INTERACTIVE SYSTEMS Workshop 2019

Collection of scientific papers

ULYANOVSK

UlGTU

2019

The Ministry of Education and Science of the Russian Federation Russian Association of Artificial Intelligence

Ulyanovsk State Technical University (Russia)

Darmstadt University of Applied Science (Germany)

Krefeld University of Applied Science (Germany)

Varna Technical University (Bulgaria)

Brest State Technical University (Belarus)

North China University of Technology (China)

IS 2019

(Ulyanovsk, Russia, International Workshop "Interactive Systems",24-27 September 2019)

УДК 681.518(04) ББК 32.96я43 И73

Editorial board:

Peter Sosnin, Prof. (Responsible editor, Ulyanovsk State Technical

University)

Vladimir Maklaev, PhD (Ulyanovsk State Technical University)

Ekaterina Sosnina, PhD (Ulyanovsk State Technical University)

УДК 681.518 (04)

Interactive Systems Workshop 2019 (Ulyanovsk, Russia, September 24-27,

2019). − Collection of scientific papers. [Электронный ресурс] :

электронные данные. – Ulyanovsk: USTU, 2019. − 110 p.

The collection of scientific papers consists of reports presented at the

Interactive Systems Workshop held within the 14th International Conference

on Interactive Systems: Problems of Human-Computer Interactions

(September 24-27, Ulyanovsk, Russia). The main accent is focused on the

problems, tasks, models, tools and technologies that use human-computer

interaction.

ISBN 978-5-9795-1970-8 © Composite authors, 2019

© Design, USTU, 2019

CONTENTS

LIMPID AND UNPERTURBED DECENTRALIZED APPLICATION FOR CROWDFUNDING USING BLOCKCHAIN TECHNOLOGY……... 5

Manoj Athreya A., Ashwin A. Kumar, Abhishek M. Holla, Nagarajath S. M. and Gururaj H.L. MEDCHAIN: SECURING ELECTRONIC MEDICAL RECORDS WITH A PEER TO PEER AND DISTRIBUTED FILE SYSTEM…………………. 17

Gururaj H.L. and Ramesh B. COMPARATIVE ANALYSIS OF NETWORKS ARCHITECTURES FOR FEATURE EXTRACTION FOR EMOTION RECOGNITION IN SOUND? ………………………………………………………………………... 33

Ilia Sedunov and Anastasiya Popova USING NEURAL NETWORK MODELS FOR CLASSIFICATION OF SHORT TEXT MESSAGES…………………………………………………... 39

Maxim Dli and Olga Bulygina TELEPRESENCE OR VIDEO CALL? WHICH IMPROVES THE WAY WE COMMUNICATE? ……………………………………………………..... 45

Rivosoaniaina Alain Nimbol, Mahatody Thomas and Josvah Razafimandimby THE MODEL OF SPECIAL COMPUTER INTERFACE FOR LEARNING ADULT STUDENTS……………………………………………. 57

Inna Bashmakova METHOD OF INCREASING THE ACCURACY OF MEASURING SIGNALS WITH THE USE OF COMBINED TEST ALGORITHMS……. 64

Almaz Mehdiyeva and E.K. Mehdizade CONSTRUCTIVE & FUNCTIONAL REPRESENTATION OF ENGINEERING PRODUCTS IN A CAD-SYSTEM AT THE STAGE OF TECHNICAL DESIGN………………………………………………………... 71

Denis Tsygankov, Alexander Pokhilko and Ivan Gorbachev IDENTIFYING SCIENTIFIC CONSTRUCTS IN THE RESULTS OF QUESTION-AND-ANSWER REASONING TO SUPPORT PROJECT THEORIZING………………………………………………………………….. 76

Anna Kulikova and Ekaterina Trifonova FORMATION OF REQUIREMENTS TO THE MODELING ARCHITECTURE OF THE AUTOMATED SYSTEM…………………….. 82

Alexandr Podobriy

ADJUSTMENT OF COORDINATE SYSTEM USING SKELETON MODELS OF PARTS AND ASSEMBLING UNITS OF AIRCRAFT PIPELINES………………………………………………………………….......

89

Pavel Pavlov ENHANCING THE ABILITY TO WORK WITH PRIMITIVES THROUGH LWIQA…………………………………………………………… 101

Alexandr Ivasev POTENTIAL OF COMPUTER GAMES IN SECOND LANGUAGE LEARNING…………………………………………………………………….. 105

Farida Sitdikova, Venera Khisamova, Timur Usmanov and Olga Danilova Anna Kulikova

MULTI-AGENT APPROACH TO FILL PROJECT ONTOLOGYWITH THE HELP OF REASONING TEXT…………………….………

Limpid and Unperturbed Decentralized Application for

crowdfunding using Blockchain Technology

Manoj Athreya A[ 1], Ashwin A Kumar [1], Abhishek M Holla [1], Nagarajath S M [1] and Gururaj H L [1]

Computer science & Engineering, Vidyavardhaka College of Engineering, Mysuru, India manojathreya.cs@vvce.ac.in, ashwinkumar.cs@vvce.ac.in, abhishek.holla98@gmail.com,

nagarajath.cs@vvce.ac.in, gururaj1711@vvce.ac.in

Abstract: Crowdfunding as the name relates its raising of funds from a vast number of sources. This can be an individual or an organization who merely wants to contribute for a noble cause. In layman terms, it can be referred to as funding given by an anonymous crowd with a belief that they are helping a section of the society to fight and overcome their monetary problems. This is being achieved via the Internet where people or groups raise funds to help others during a natural catastrophe or some community-oriented social projects, entrepreneurial ventures, travel, medical expenses and many more. The present-day scenario is such that once people transfer money into a particular venture no one knows what happens to the money generated. As a solution, one needs to track down the flow of money and thereby maintaining appropriate transparency. We propose an approach through this paper, of a decentralized system built using Blockchain Technology where once the investor funds money to a venture the transactions will be captured. If incase, the endeavor initiated accumulates the money within the stipulated time then it is utilized or else the money is returned back to the investor. This platform will eliminate the middlemen completely. It is transparent and more secure and thus acts like a huge savior for the people in need. A decentralized approach to crowdfunding forfeits all fees for the investor, gives the receiver more share of the project, and allows for a peer-to-peer relationship between the investor and receiver.

Keywords - Crowdfunding, Blockchain, Smart Contracts, Peer-to-Peer Network, Internet.

1. Introduction

Crowdfunding refers to a project meant for gathering funds. This can be for variedreasons ranging from as severe as a natural calamity or may be small amounts of money to help people for their work or projects through the Internet[1]. The present model of crowdfunding solely revolves around three kinds of players they are projected initiator one who suggests the idea, people or group of people who fund the project and organization that is responsible for bringing parties together[2]. In the system, interested people or collective investors who fund the project are led to the list of initiated projects appearing on the dashboard. The project initiator creates a project by giving its description and linking his metamask account to the project. The organization keeps track of all the activities done in the system. This can be divided into four categories: reward crowdfunding, social crowdfunding, equity crowdfunding,

and peer-to-peer funding. Social and reward crowdfunding is raised for charitable causes [10]. Equity crowdfunding is a technique of raising capital through selling of stocks to a number of individual investors using the equivalent method as crowd-funding [3]. Peer-to-peer funding refers to the use of an online platform to unite the lenders/investors with borrowers/issuers in order to provide unsecured loans.

It serves as a way to overcome the financing gap in the primitive stages of a novel

project. Funding from venture capitalists and banks is usually available only in the

later development phases of start-ups [4]. In the initial periods of a company’s development cycle, a large amount of the fund is contributed by the founder himself or to some extent by his friends and family. If incase these funds are insufficient, the project faces a funding gap [5, 9]. The investors are not just financially motivated but social reputation and intrinsic motives play a significant role. The motives of participants in crowdfunding are different and may also depend on the model adopted. Due to the rapid growth of social media, it is now identified as the key for the capital providers to participate in crowdfunding platform [6]. Experimental results have also shown that social media reduce most of the information asymmetries and hence, enhance the possibility of funding. The existence of such a platform serves as a major advantage for both investors and initiators. In order to provide a standardized process, the platform acts as an information, communication and secure portal [7, 8].

The system proposed is secure as the transactions are captured onto the blockchain. The venture is termed as successful if the initiated project collects the designated amount in the stipulated time else it will be deemed as a failure and through backtracking process, the money will be returned to the investors. Before initiating the project, the necessary documents are furnished and on successful verification, the project is launched. Thus making it secure and transparent to the investors.

This paper provides an insight into the existing platforms in crowdfunding and the updated proposed system making the platform safe, secure and more transparent. The focus of our study lies in the decentralized approach to the existing system. The research work is done on this platform with an aim to eliminate the middle-man and to reduce the swindling activities in this domain. Through approaches like the aforementioned, startups will get more opportunities and a better platform to build and capitalize on the market.

In the next section, we will discuss the literature survey done in this field. Section 3.0 deals with the background and existing system. Section 4.0 deals with the terminology used in the project. Section 5.0 deals with case study analysis. Section 6.0 deals with result analysis followed by conclusion and references.

2. Literature Survey

Agrawal, A. [1] et.al proposed research work based on geographic location, the influence of social media and time to fund the project and analysis on it. This also gives an overview of the financial aspects involved in crowdfunding platform and their transactions. Ahlers [2] et.al proposed their work on the equity crowdfunding

6

platform in increasing its performance and providing equal shares to the investor and initiator of the project based on their ECP algorithm. Aitamurto T [3] et.al proposed their work on the impact of crowdfunding platform on journalism and its practices which showed a new model of approach to the existing system. Davis B.C [5] et.al proposed their work on examining the role of intrinsic versus extrinsic cues. Entrepreneurship Theory and Practice based on the funding issues faced in the initial process of beginning the company. Quercia [6] et.al proposed a system for the investors by building a recommender system so that the investors are guided about the project so that they get maximum shares for the funds they invested. The recommender system takes into account the past and present scenario of the project and its details for recommending. Ashta [7] et.al carried an analysis of the investors investing in crowdfunding projects on European online micro-lending website and found that the funds were more to Innovative marketing projects. Mahagaonker [8] et.al proposed a financial signaling system which used to send updates to the investors regarding a new project in the crowdfunding platform which is innovative and more profitable. Bachmann [11] et.al proposed a literature review on Online peer-to-peer lending system and the working of internet banking and commerce in the centralized architecture. Hekman [14] et.al proposed Social network analysis on the relationship between the success of crowdfunding projects, social networks of initiators and media activities.

3. Background and Existing System

Of late the term crowdfunding has got a lot of steam. This is due to the huge number of online campaigns to raise money for various causes. In simple terms, this could be interpreted as an individual or a group introduces a notion or situation that may require financial support. At the core, these benefits startups, personal projects, etc the most. However, there is a lot of uncertainty encircling it.

They opt for this kind of a system for varied reasons ranging from a natural calamity to help people for their work or projects. Big companies like paytm, facebook, etc have stepped into this field and thereby provide a platform for donating money for a noble cause. But the biggest question that surrounds this is that nobody knows whether the money donated is reaching the needy or is being manipulated at some point.

To mitigate this, Blockchain Technology can be used as once the transaction details are stored on a block it cannot be altered. If anyone tries to change even a small character immediately the hash value changes due to the avalanche effect and hence indicating that the data has been manipulated. This provides high security of data and eliminates the manipulation of information. Moreover, the investor can keep track by using the hash value generated when the transaction details are mined into a block as data. The advantages of this kind of system are it is more transparent, secure and manual errors are eliminated.

7

4.Related Terminologies

Blockchain is one of the booming words in the field of Computer Technology, which has the power to change the lives of people as the Internet did in the past twenty years. It is ready to make a big impact on the lives of people if we adhere to this technology. As it is a tech-savvy concept containing many technical terms, some of them which are required to understand are listed below

4.1 Blockchain

Blockchain can be best described as an expanding list of records called blocks, linked and stored using cryptography. The primary block is attributed as the genesis block. All blocks will have the succeeding details in it:

1) Data: String of characters stored.2) Nonce: A unique number related to mining.3) Previous Hash: Hash value of a block that came before the current block.

This field establishes the cryptographic link with the following block.4) Hash: Fingerprint for the data stored in the block.

4.2 Ethereum

Ethereum is a dynamic, decentralized, open-source service that operates on the characteristics of the blockchain. It was originally proposed in 2013 through a white paper by Vitalik Buterin. This was obtained from the bitcoin project which is fundamentally a tool intended towards monitoring transactions among people. The essence of ethereum gyrates around smart contracts. These are small blocks of code residing in the blockchain intended to perform a specific task. Ethereum can be described through the following:

1) Ethereum NetworkThe base of a decentralized network is made up of an assortment of nodes

interacting with each other. This is largely associated with the transfer of money and storage of data, achieved through a cryptocurrency called ether. Ether is very much similar to bitcoin and is accountable for fueling the ethereum network.

2) Interfacing with EthereumInterfacing refers to the process of interacting with the network. This can be accomplished in 2 ways:

a) Web3 library: It is an API predominantly used by the developers tointeract with the network. b) Metamask: It is a browser extension used by users to interface with thenetwork. 3) Ethereum Account

An ethereum account will have the following basic elements: a) Balance: The number of ethers owned by this address.

8

b) Nonce: It depicts the number of transactions carried out from a specificaccount address or illustrates the number of contracts produced by theaccount.

c) CodeHash: Hash code of the EVM of this account.d) StorageRoot: A 256-bit value representing the encoded version of the

data stored in the chain.

4.3 Dapp

Dapp is an open-source application that works totally on the smart contract code running on the blockchain. These have been designed in such a way that, it is not controlled by any single entity but rather regulated by blocks of code known as smart contracts. DApps uses decentralized storage to store data and code. The aforementioned is a blockchain-based app, where the Smart Contract is employed to connect to the blockchain[15].

5. Mathematical Analysis

The main objective of the below-mentioned algorithm is to protect our systemfrom all failures. This works on the consensus protocol. All the nodes in the network, c reate an indisputable system of understanding among various nodes in a distributed network.

Algorithm BFT (for nodePi ) :

Let A = Ω(λN2 logN) be the batch size parameter. Let PuK be the public key received from TPKE. Setup (executed by a dealer), and let Si be the secret key for Pi . Let buffer: = [ ] be a FIFO queue of input transactions.

Proceed in consecutive epochs numbered r:

// Step 1: Random selection and encryption •let proposed be a random selection of [A/N] transactions from the first N elements ofbuffer •encrypt y := TPKE.Enc(PuK,proposed)// Step 2: Agreement on ciphertexts •pass y as input toACS[q]•receive vj j∈G, where G ⊂[1..N], from

9

ACS[q] // Step 3: Decryption

•for each j ∈ G:Let ej : = TPKE.DecShare(Si , vj ) multicastDEC(q,j,i,ej ) wait to receive at least f+1 messages of the form DEC(q,j,k,ej ,k) Decode xj : = TPKE.Dec(PuK,(k,ej ,k) )

•let blkr : = sorted(∪j ∈S yj ), such that blkr is sorted in acanonical order (e.g., lexicographically) •set buf := buffer−blkr

6. Case Study Analysis

The decentralized platform is built utilizing ethereum tokens. The investor funds aproject by transferring tokens from his metamask account to the project initiator account. Thus storing all the transaction details onto a block of a blockchain. A project is successful if it accumulates the expected amount within the specified time otherwise is regarded as a failure and the investor gets back the money through the backtracking method.

Figure 5.1

The above figure shows the index page of the crowdfunding project where an initiator can start a venture of raising funds.

10

Figure 5.2

The above figure depicts the basic information collected for creating a venture by the initiator. And on verification, the project will be launched on the website for investors to fund.

Figure 5.3

11

The figure illustrates how the account details of initiator are verified before creating and starting the project online by the system.

Figure 5.4

The above figure portrays the dashboard of the website. This displays all the initiated projects with the newest project first so that investors find it easy to fund.

12

Figure 5.5

The figure demonstrates how an investor can fund a project by sending ethers directly to initiator account with the help decentralized platform.

Figure 5.6

13

The figure demonstrates how the backtracking method gives an option to the investors to transfer the ethers directly into their account.

6. Result Analysis

Today, the transaction is accepted on the Ethereum network by paying 21,000 gasand sending a 20 gwei/gas fee.

With the average block time currently about 14.5 seconds, whenever a transactionis submitted to the network it takes about 53 seconds to be confirmed by a miner and added onto a block in the blockchain. The Four main factors that hold up our standard transaction from being included in a block are:

1. Block interval time2. Proof Of Work mining software3. Gas price4. Empty block penalty.

Figure 6.1

Above result shows the minimum, maximum and average gas used for each function that is being written on the blockchain where the gas value is 21 gwei per gas. It also specifies the number of calls it makes to the chain and average total cost for each function. Thus for deploying a smart contract on the ethereum blockchain a minimum of 4.23 USD is required.

7. Conclusion

In this paper, we have proposed a decentralized platform for crowdfunding wherewe eliminate the middle agents and fake people responsible for dismantling the system. The results and proposed work shows the effective working of the system

14

making it more secure, easy and transparent system compared to the existing centralized, middle-man system. The project supports the initiators through a simple and secure GUI and thereby ensuring that they launch there project easily. It also helps investors to easily fund projects and the transaction of ethers are stored in a blockchain for integrity purpose. It also backtracks the transaction if the project initiated fails to raise funds within the stipulated time, thus forming a trustful system which the investors can rely upon. From the above-obtained results, the system is much more accurate and secure medium of raising funds.

References

1. Agrawal, A., Catalini, C., & Goldfarb, A. (2014). Crowdfunding: Geography,social networks, and the timing of investment decisions (Working Paper).Retrieved July 20, 2014, from http://www. 487 catalini.com/s/Crowdfunding_Geography_Social_Networks_2014_05_27.pdf*

2. Ahlers, G., Cumming, D., Gu¨nther, C., & Schweizer, D. (2013). Equitycrowdfunding (SSRN) Working Paper No. 2362340). Retrieved May 15,2014, from http://papers.ssrn.com/sol3/ papers.cfm?abstract_id¼2362340*

3. Aitamurto, T. (2011). The impact of crowdfunding for journalism.Journalism Practise, 5(4),429–445. doi:10.1080/17512786.2010.551018*

4. Allen, F., & Santomero, A. (1997). The theory of financial intermediation.Journal of Banking & Finance, 21(11–12), 1461–1485. doi:10.1016/S0378-4266(97)00032-0.

5. Allison, T. H., Davis, B. C., Short, J. C., & Webb, J. W. (2014).Crowdfunding in a prosocial microlending environment: Examining the roleof intrinsic versus extrinsic cues. Entrepreneurship Theory and Practice.doi:10.1111/etap.12108*

6. An, J., Quercia, D., & Crowcroft, J. (2014). Recommending investors forcrowdfunding projects. 499 In WWW’14 Proceedings of the 23rdInternational Conference on World Wide Web. International World WideWeb Conference Steering Committee (pp. 261–270). doi:10.1145/2566486.501 2568005*

7. Ashta, A., & Assadi, D. (2010). An analysis of European online micro-lending websites. Innova- 503 tive Marketing, 6(2), 7–17. Retrieved fromhttp://businessperspectives.org/journals_free/im/5042010/im_en_2010_2_Ashta.pdf*

8. Audretsch, D. B., Bonte, W., & Mahagaonkar, P. (2012). Financial signalingby innovative nascent 506 ventures: The relevance of patents and prototypes.Research Policy, 41(8), 1407–1421. 507 doi:10.1016/j.respol.2012.02.003.

9. Bachmann, A., Becker, A., Buerckner, D., Hilker, M., Kock, (2011). Onlinepeer-to-peer lending—A literature review. Journal of Internet

15

Banking and Commerce, 16(2). Retrieved from http://www.arraydev.com/commerce/JIBC/2011-08/Alexander Becker.pdf*

10. BaFin. (2012, September). BaFinJournal. Retrieved fromhttp://www.bafin.de/SharedDocs/Down514loads/DE/BaFinJournal/2012/bj_1209.html

11. Barasinska, N., & Scha¨fer, D. (2010). Does gender affect funding success atthe peer-to-peer credit 516 markets? (DIW Berlin Discussion Papers No.1094). Retrieved July 15, 2013.

12. Barasinska, N., & Scha¨fer, D. (2014). Is crowdfunding different? Evidenceon the relation between 519 gender and funding success from a german peer-to-peer lending platform. German Economic Review.doi:10.1111/geer.12052*

13. Baum, J. A. C., & Silverman, B. S. (2004). Picking winners or buildingthem? Alliance, intellectual, and human capital as selection criteria in venturefinancing and performance of biotechnology startups. Journal of BusinessVenturing.

14. Hekman and Brussee (2012). Crowdinvesting: Die Invest Platform, P., &Lambert, T. (2014). Crowdfunding: Some empirical findings andmicroeconomic underpinnings (SSRN Working Paper No. 2437786).

15. Vitalik Buterin, “A NEXT GENERATION SMART CONTRACT &DECENTRALIZED APPLICATION PLATFORM”, Ethereum White Paper.

16

Medchain: Securing Electronic Medical Records with a Peer to Peer and Distributed File System

Gururaj H L

Computer Science and Engineering Vidyavardhaka College of Engineering, Mysuru, India

Ramesh B

Computer Science and Engineering Malnad College of Engineering, Hassan, India

Abstract. Though the advances in technology have changed every aspect of our lives, it has barely budged the way we think about our medical records being stored across various platforms. This electronic document talks about the problem that has always existed in the medical field but has failed to be solved; Privacy in the current electronic medical record systems. This paper discusses the failure of traditional medical record systems in terms of security and sustenance to ease of use and provides a possible solution to use today’s technology to address the disadvantages.

Keywords. EHR, health, privacy, medical, records, security, IPFS, bigchaindb, RFID, dApp, distributed systems, EAS

I. Introduction

The amount of digital data over the last few years have increased exponentially to the point where it doubles every year and completely changing how we live every day. This shows that there is no doubt that the oil of the future economy, is data. Big companies are in need of more and more data as machine learning algorithms become more robust. Social media platforms like Facebook are already facing the backlash for selling data. In the future, companies and big corporate may even pay to get their hands on our data, as data is valuable intrinsically [1].

Today, implementing, maintaining and upgrading their electronic health record systems posed as a major challenge to hospitals and health systems [2]. The theme of this document is to securely store health records and maintain a single version of the truth. A probable solution is to convert to a decentralized application. The different organizations such as doctors, hospitals, laboratories and other health insurers can record transactions and serve their purpose, on the distributed ledger, by requesting permission to access a patient’s record. Electronic health records can be stored and shared securely by building a platform and creating a distributed access with a validation system which will help to completely replace the current centralized intermediaries. Thus providing a solution to today’s health record problems.

II. Traditional electronic health record systems

Over the past few years doctors, nurses and health professionals are limited in the level of care that they can provide. This is due to the inability to view the complete

2017 has seen the most data breaches than any other year. Between the years 2009 and 2017, there have been 2,181 health data breaches. These breaches were involving more than 500 records which resulted in the theft or exposure of up to 176,709,305 health records. This is almost and accurate health record. This paper ignores the existence of non-electronic records and only considers existing Electronic Health Records (EHR). An electronic health record (EHR) is a record of details patient's medical health like physical examination, history, investigations and treatment stored in a digital format. There is a different Record Management Software for every hospital. Some use a cloud service provider, some store data locally in their databases and some store the data in a format compliant with insurance agencies. The user’s data is on a server that belongs to the hospital or is rented by the hospital most of the time [3].

A. Major Problems caused by this Model [4]

• Fragmentation of the patient’s medical information. As patientsmove between providers, they lose easy access to past records astheir data becomes scattered across different organizations. Thiscauses fragmentation of the patient’s medical information acrosshospitals, private medical practitioners, and other m-health apps.

• Transferring records from one hospital or application to another is achallenge. Patients are not provided full access to their healthrecords. Hence they have to get multiple tests done multiple timesacross multiple organizations.

• Inability to access vital medical information, in case ofemergencies.

• Data leaks from hospitals that sell the patient data to companiesthat benefit from patient’s information.

• Data manipulation can be done by hospital authorities.• Unauthorized access to the patient’s medical data.

III. Statistics

The Department of Health and Human Services’ Office for Civil Rights gave a statistics from October 2009 of healthcare data breach as shown in Fig.1. Only data breaches of 500 or more records are included in the statistics. Breaches still being investigated by OCR, as well as closed cases are included in these statistics. According to the statistics over the past 9 years, there has clearly been an upward trend in data breaches. According to the records, upto 54.25 percent of the population of the United States. These data breaches in healthcare are now being reported to be more than one per day.

18

Figure 1. Number of Reported Data Breaches (2009-2018)

More healthcare data breach statistics showed in Fig.2 and Fig.3. These statistics show that, hacking is still the leading cause of healthcare data breaches, even though health-care organizations now are much better at detecting breaches. The low hacking incidents in the earlier years is probably due to the organization’s inability to quickly detect malware infections and hacking incidents. Many of the hacking incidents between the years 2014 and 2017 occurred for many months or also sometimes, years, before they were detected.

Healthcare organizations are recently getting better at detecting internal breaches and are reporting these breaches, in time, to the Office for Civil Rights. Although hacking is currently reported to be the main cause of breaches, unauthorized access to healthcare records or disclosure incidents are catching up and are in close second.

Figure 2. Hacking/IT incidents Graph

19

Figure 3. Records Exposed due to Hacking/IT incidents

Figure 4. Unauthorized Access/Disclosure Incidents Graph

No incident that has occurred can be trated lightly as the impact of each incident as seen in Table I. must be dealt with seriously. Data breach and impractise in the medical field is the latent issue that we need to address at the earliest.

Table 1. Largest healthcare data breaches

Name of Covered Entity

Year Covered Entity Type

Individuals Affected

Type of Breach

Anthem Inc 2015 Health Plan 78,800,000 Hacking/IT Incident

Premera Blue Cross 2015 Health Plan 11,000,000 Hacking/IT Incident

20

Excellus Health Plan Inc. 2015 Health Plan 10,000,000 Hacking/IT Incident

Science Applications International Corporation

2011 Business Associate

4,900,000 Loss

University of California, Los Angeles Health

2015 Healthcare

Provider

4,500,000 Hacking/IT Incident

Community Health Systems

Professional Services Corporations

2014 Business Associate

4,500,000 Hacking/IT Incident

Advocate Medical Group

2013 Healthcare

Provider

4,029,530 Theft

Medical Informatics Engineering

2015 Business Associate

3,900,000 Hacking/IT Incident

Banner Health 2016 Healthcare

Provider

3,620,000 Hacking/IT Incident

Newkirk Products, Inc

2016 Business Associate

3,466,120 Hacking/IT Incident

With the ever greed in the race towards developing technology, the rise in better systems and better ways to breach security arises. The development of electronic medical records can no longer be stagnant and needs a revolutionary improvement to challenge today’s needs[5].

Figure 5. Rise in breaches vs Improvements in EHRs

21

IV. Strategy

As shown in Fig. 5, the number of breaches seems to be increasing or at least the possibilities of another huge data breach but the improvement needed to combat such risks in EHRs have completely been stagnated. The use of latest technologies to combat latest risks is an ideal option.

A. IPFS – Inter Planatary File System

Traditional Client-Server architecture based applications come with a lot of disadvantages. Traffic congesion is one of the major problems. Problems can occur when a large number of simultaneous clients send requests to the same server. The robustness of a network P2P is absent in the paradigm of Client-Server classic. Customer requests cannot be met when the servers go down. But P2P network resources are distributed across multiple nodes of the network. A Client Server system also comes with a very high expense as the security, robustness and clients increase and hence the server needs to scale up [6].

Medical records are hence chosen to not be stored in a Client Server model. The idea of storing all the records in a single location not only compromises the security aspect of the application but also increases the cost to set up a server to serve multiple requests to store and fetch files of variable sizes. Hence we choose to use IPFS.

In HTTP, a file is downloaded from a single computer at a time, instead of simultaneously getting different pieces from multiple computers. A P2P approach will be able to save 60% in bandwidth costs, with video delivery.

Distribution of high volumes of data, with high efficiency can be achieved using IPFS. Zero duplication leads to massive savings in storage. The fact that the average lifespan of a web page is only 100 days is not good enough, as the primary medium of our era cannot be so fragile. Every version of your files is stored by IPFS and it makes it simple to set up resilient networks for mirroring of data.

IPFS is currently the original vision of the open and flat web. The IPFS delivers the technology which makes that vision, a reality.

Offline, intermittent connections, natural disasters, developing world, are all trivial when compared to the interplanetary networking. The networks used today mostly belong to the 20th Century. The creation of diversely resilient networks is powered by IPFS. With or without Internet backbone connectivity, IPFS enables persistent availability.

IPFS has a very unique way to store the files that is provided. A unique fingerprint is given to each file and all the blocks within it, called a cryptographic hash. Next IPFS makes sure that there are no duplicate files receding in the network, rather than finding duplicates with merely each file’s name, IPFS checks for the content and removes duplicate content. To improve efficiency, IPFS nodes only store content, that it is interested in and a little indexing information about the files that are being stored. Now when a file is requested from the IPFS a query with the hash is sent which is unique to each content of the file within a network.

IPFS is hence the best way to store the medical records for its security, P2P system and low cost to set up a node. Medchain uses the ipfs-http-client library which exposes multiple APIs to interact with the IPFS.

22

B. BigchainDB

BigchainDB is like a database with blockchain characteristics, with high throughput, low latency, powerful query functionality, built-in asset support, decentralized control and immutable data storage. BigchainDB allows developers to deploy blockchain proof-of-concepts, platforms and applications with a blockchain database, that helps it in supporting a wide range of use cases.

BigchainDB starts with a bigdata distributed-database and rather than trying to enhance blockchain technology, it adds blockchain characteristics.

• Decentralization – No single point of control via a federation of votingnodes makes for a P2P network.

• Immutability – More than just tamper-resistant. data can’t be changed ordeleted once it is stored.

• Byzantine Fault Tolerant – Even if up to one third of the nodes in thenetwork are experiencing arbitrary faults, the rest of the network will stillcome to consensus on the next block.

• Customizable – Design a completely custom private network withtransactions, custom assets, transparency and permissions.

• Open Source – Any developer can use it and build their own applications ontop of it.

• Query – Write and run MongoDB query to search the content of all storedtransactions, assets and metadata.

• Native Support for Multiassets – Any asset token, or currency can be used,due to absence of a native currency.

• Low Latency – A global network takes about a second to come to consensuson a new block, i.e., transactions happen extremely fast.

The IPFS exposes a cryptographic hash to refer the files that are stored in its system. Anybody who can access to his hashes have a possibility to access the file from the IPFS network. Medchain therefore required a system to store and record these assets in an immutable system. BigchainDB is the optimal solution. The hashes once generated are stored in BigchainDB and recorded as assets with the owners of each file via their public keys and signed using their private keys.

C. MongoDB

As Medchain is an end to end application, there exists data which needs to be securely stored but cannot use a decentralized system or blockchain. For example, an authentication system needs the username and password to be stored, but a blockchain or IPFS cannot be used because of the toll it takes on the user interface of the application. It was therefore required for a substitute storage solution for all other local information and to store each account’s cryptic public key. Hence MongoDB was the obvious choice, they come with the following characteristics [7].

• Dynamic schema: This gives you flexibility to change your data schemawithout modifying any of the existing data.

• Scalabilty: MongoDB is horizontally scalabale, which helps scale yourbusiness and reduce the workload with ease.

23

• Manageability: The database is fairly userfriendly since doesn’t require adatabase administrator. It can be used by both developers and administrators.

• Speed: It’s high-performing for simple queries.• Flexibility: New columns and fields can be added without affecting existing

rows or application performance.

D. RFID

RFID or Radio Frequency Identification System is a technology based identification system. The RFID helps identify objects just through the tags attached to them. They do not require any light of sight between the tags and the tag reader. The only thing that is necessary is a radio communication between the tag and the reader.

There are three main components of a RFID system: a RFID tag, a reader and a processor that receives the reader input and processes the data.

RFID tags are best suited to be linked with individual patients and a reader can be installed and set up with a doctor. The patient can carry the tag rather than a bulky and fragile medical file and trust Medchain to handle all the data linked with the patient. E. TENDERMINT

The tendermint core is a byzantine-fault tolerant state machine replication system or a blockchain for short. The Tendermint Core is a application platform of the blockchain . Tendermint provides the equivalent of a database, or a web-server, or supporting libraries for blockchain applications which can be written in any programming languages. Tendermint serves blockchain applications, just like a web-server serving web applications.

Tendermint Core performs Byzantine Fault Tolerant (BFT) State Machine Replication (SMR) for arbitrary deterministic, finite state machines.

Even if up to 1/3 of the machines fail in arbitrary ways Tendermint can work. Also, every non-faulty machine sees the same transaction log and computes the same state. A fundamental problem in distributed systems is having secure and consistent replication. It plays a critical role in the fault tolerance of a broad range of applications.

The of two chief technical components of Tendermint are a blockchain consensus engine and a generic application interface. The Tendermint Core, or the blockchain consensus engine, ensures that the same transactions are recorded on every machine in order. The application interface, called the Application BlockChain Interface (ABCI), allows the transactions to be processed in any programming languages. Medchain does not directly interact with the Tendermint Core but uses the driver provided by BigchainDB to do the same.

24

V. Architecture and implementation

The idea is to make Medchain a web application rather than the traditional native windows application because it is easy to set up and runs on any system which can run a browser.

Fig. 6. Shows the architecture of Medchain, the medchain symbol depicts the API which interacts with various elements of the architecture. Medchain is mainly built using the MERN stack.

Figure 6. Medchain Architecture

25

The MERN stack consists of Express, MongoDB, Node.js and React/Redux. The MERN stack is one of the most popular stack of technologies that can be used for building a modern single-page web applications, while keeping in mind the success of React in the frontend and of Node.js on the backend.

The application mainly follows a particular flow, that patient, the doctor, the lap technician and any other third party can access Medchian. Once inside, any party has the ability to add records and connect it to a patient. The patient then has the ability to give permission to any party that needs the access. The patients can also store important emergency information which can be accessed via RFID cards.

The Medchain API interacts with MongoDB using mongoose to store non-sensitive information. Mongoose is an Object Data Modeling (ODM) library for MongoDB and Node.js. Mongoose can be used to translate between objects in code and the representation of those objects in MongoDB. It also manages relationships between data and provides schema validations. Mongoose exposes various APIs to make it easy to interact with mongodb. Many functions like find(), updateOne(), etc, can be used via mongoose. The application uses three models: File, Transaction and User to interact with the particular collections on Mongodb.

Medchain also uses Express to make RESTful APIs. Express is a unopinionated, fast, minimalist web framework for Node.js. Express and other nodejs packages are installed via npm or Node Packages Manager. Express is also installed the same way. But mainly, express is a flexible Node.js web application and aminimal framework that provides a robust set of features for web and mobile applications. With a ocean of HTTP middleware and utility methods, express is capable of creating robust APIs that are quick and easily. Without obscuring Node.js features, Express provides a thin layer of fundamental web application features.

Medchain API interacts with the IPFS using ipfs-httpclient. This is basically a client library for the IPFS HTTP API, implemented in JavaScript. This client library implements the interface-ipfs-core enabling applications to change between an embedded js-ipfs node and any remote IPFS node without having to change the code. In addition, this client library implements a set of utility functions. The two major functions that are used are ipfs.add() and ipfs.get() which are mainly used to store the files that is passed from react and served on Node.js via express and axios. The add function uses the file that is obtained using formidable, a Node.js library to handle files that are sent over an API. These files are then published to IPFS which returns the hash as a result. This hash is then used to get the file using the ipfs.get() to obtain the file.

Medchain interacts with Tendermint via BigchainDB. BigchanDB is used in Node.js using the js-bigchaindbdriver. The main operations that are involved in Medchain and BigchainDB are the file information and the transactions. The file part keeps in check the file and the owner of the file. The public key is used to verify the file information and the private key is used to sign the asset. In the same way Medchain also records transactions and records them as an asset after getting signed from the owner of the transaction.

The patient obviously cannot expect to interact with Medchain during emergencies and hence are required of a system to overcome this. RDIF provides a viable solution where the important information is stored on MongoDB and the scanner can scan the IDs and process the information.

26

Finally, the application uses React for the front-end single page application. JavaScript library for building user interfaces is React. React makes it painless to create interactive User Interfaces. Medchain is completely built on React and hence has a robust architecture. The frontend interacts with Medchain API to use all its functions via axios.

VI. Results

The users can be generally divided into patients and Doctors/Lab technicians.

Figure 7 shows the login page for Patients, which requiers the patients phone number.

Figure 7. Login Page

Figure 8. Patient Registration Page

27

patient and the doctor/ lab technician respectivey. The new patient is requiered to register by entering their name, phone number and a password, while the doctor/ lab technician can register by entering name, usernamme and password as their credentials.

Figure 10 shows the Dashboard of a user, containing the nummber of files presenting in the users account and the number of transactions that have been made with respect to any file belonging to the user.

Figure 9. Doctor Registration Page

Figure 10. New Dashboard

28

Figure 11. Add Files

The Patient or the Doctor can add files into the patient’s account as shown in figure 11 and figure 12. The doctor can enter the patients number to let the platform know which patient is to receive the files.

Figure 12. Add files to a patient

Figure 13 . Upload Files

29

The uploaded files will be displayed in the window as shown in figure 13 and the number of Files in the patients dashboard gets increased accordingly. The transaction count increases in both the patient’s and the doctor’s dashboard.

The doctor cannot view any of the patient’s files untill the patient permit the doctor to view a file. As shown in Figure 14 the patient has to enter the user name of the doctor/lab technician to give them access to the file selected.

Figure 15 contains the updated dashboard of the doctor which shows the increase in the number of files and transactions. The doctor now also has the permission to view the patients file.

Figure 14. Permit ThirdParty

Figure 15. Updated Dashboard

30

Figure 16 shows the list of nodes that the user’s node is connected to, in a network. The location and ip of each connected node is also displayed.

VII. Conclusion

The current EHR systems have major drawbacks due to the possibilities of fragmentation of data, manipulation of data, data leaks and inability to access vital information in cases of emergencies. This traditional system of EHRs can be challenged by using the latest technologies to combat the latest risks.

The use of IPFS in Medchain increases security and reduces expenses due to the low cost for setting the node. BigChainDB acts like a database with all of Blockchain’s abilities making it easy for developers to deploy blockchain proof-of-concepts, platforms and applications with a blockchain database, support a wide range of use cases. The use of Tendermint makes Medchain less affected to machiene failures. Tendermint works even if up to 1/3 of the machines fail in arbitrary ways.

By using medchain the fragmentation of data is avaoided, privacy and security can be maintained efficiently. The patients will be have complete access and control over their data and will also have the capability to provide access to various users, hence improving data security.

In case of emergencies the rik of inability to access information is irradicated as the RFID cards carried by patients can be scanned by any medical practitioner, which will let them easily acquire the vital information about the patient’s health.This will eradicate the problems of the current Electronic Health Record (EHR) systems. By digitizing health records and empowering users countless industry problems can be reduced.

Figure 16. New Dashboard

31

Refernces 1. Jyoti Nigania, Data is Gold: The Most Valuable Commodity, house of bots, May

15, 2018.2. Menachemi N, Collum TH. Benefits and drawbacks of electronic health record

systems. Risk Manag Healthc Policy. 2011;4:47–55.doi:10.2147/RMHP.S12985I.

3. Aickin M. Patient-centered research from electronic medical records. Perm J.2011;15(4):89–91.

4. Stephen O’Connor, Pros and Cons of Electronic Health Records, Advanced DataSystems Corporation, February 6, 2017.

5. HIPAA Journal, Healthcare Data Breach Statistics.6. ESDS, Advantages and Disadvantages of Client application server, January 7,

2011 7. Xplenty, The SQL vs NoSQL Difference: MySQL vs MongoDB, Medium,

September 28, 28. Debra Bradley Ruder. Malpractice Claims Analysis Confirms Risks in EHRs.

February 9, 2014 ‐ Patton McGinley.9. Melissa Steward, Electronic Medical Records Privacy,

Confidentiality, Liability.10. Satoshi Nakamoto. Bitcoin: A Peer-to-Peer Electronic Cash System.11. Josh Benaloh, Melissa Chase, Eric Horvitz, Kristin Lauter. Patient controlled

encryption: ensuring privacy of electronic medical records. Proceeding CCSW'09 Proceedings of the 2009 ACM workshop on Cloud computing security.Chicago, Illinois, USA — November 13 - 13, 2009.

12. Wei-Qi Wei, Cynthia L Leibson, Jeanine E Ransom, Abel N Kho, Pedro JCaraballo, High Seng Chai, Barbara P Yawn, Jennifer A Pacheco,and Christopher G Chute. Impact of data fragmentation across healthcare centerson the accuracy of a high-throughput clinical phenotyping algorithm forspecifying subjects with type 2 diabetes mellitus. J Am Med Inform Assoc. 2012Mar-Apr; 19(2): 219–224. Published online 2012 Jan 16.

13. Michael Crosby, Nachiappan, Pradhan Pattanayak, Sanjeev Verma, VigneshKalyanaraman. BlockChain Technology Beyond Bitcoin.

14. Zainab Alhadhrami, Salma Alghfeli Mariam Alghfeli, Juhar Ahmed Abedlla,Khaled Shuaib. Introducing blockchains for healthcare, 11 January 2018.

15. Gajendra Jung Katuwal, Sandip Pandey, Mark Hennessey, BishalLamichhane. Applications of Blockchain in Healthcare: Current Landscape &Challenges. December 2018.

32

Comparative Analysis Of NetworksArchitectures For Feature Extraction For

Emotion Recognition In Sound?

Ilia A. Sedunov[0000−0002−3189−6634]

Anastasiya S. Popova[0000−0002−4650−3522]

National Research University Higher School of Economics, Nizhniy Novgorod,Russian Federation itnn@hse.ru

https://nnov.hse.ru/en/

Abstract. In this paper we describe an emotion recognition experi-ments in the audio signal, reducing to the problem of image classifi-cation. For visualization of audio-signal, we used images represented bymelspectrograms. Experiments was based on SAVEE dataset, which in-cludes 15 sentences for each 7 emotion categories: ’anger’, ’disgust’, ’fear’,’happiness’, ’neutral’, ’sadness’ and ’surprise’. We presented a compar-itive research in feature extraction with various CNN [1] architecturesand LSTM [2] applyed to the melspectrograms and the MFCC [3] co-efficients and came to a decision that the most significant features foremotion recognition can be extracted by DenseNet [4] and AlexNet [6]structures.

Keywords: Deep learning · Classification · Convolutional neural net-works · Audio recognition · Emotion recognition · Speech recognition

1 Introduction

Nowadays task of emotions recognition one of the most progressive areas in Com-puter Science. The vast majority of corporations, which use interaction betweensystems and people, can recognize human speech, and emotion classificationplays a great role in this process. Solving this problem allows us to reduce thegap between human and machines and allows to get users feedbacks in a naturalway. Developing in this area will facilite communication machine and human,moreover understanding emotions have a positive impact on business and soci-ety. Vivid example is services, using for control system of Smart Home, such asApple HomeKit, Google Assistant or Amazon Alexa.

2 Materials and Methods

In the previous papers a straightforward approach was presented and resultsshowed that the best way of voice visualysing is melspectrograms. Another way of

? The article was prepared within the framework of the Basic Research Program atthe National Research University Higher School of Economics (HSE).

audio representation which can allow us to improve the quality is mel-frequencycepstral coefficients (MFCC) [3]. In this paper we consider a comparitive researchin feature extraction for emotion classification problem using different audioflowrepresentations and network architectures.

The issue of classification can be implemeted as a creating problem of afunction construction y:X → Y , where X - is the characteristics set of objects,Y - is the finite class set. Therefore, the main task is to construct an algorithmA:X→Y, which solves the problem of classifying an arbitrary object x from theset X, where Xm = (x1, y1),...,(xm, ym). Classification is made for the imageof the correct card (y:Rn → Y), with n - the number of pixels in the image.

Latest papers has shown that CNNs demonstrate significant result improve-ment. The most valuable contribution on the start of the development of netswas made by AlexNet [6] architecture on the start of the contribution.

AlexNet [6] is CNN, which has had a great impact on development of ma-chine learning, especially on the algorithms of computer vision. Architectureof AlexNet similar LeNet [7], which was developed by Yann LeCun. However,AlexNet has more filtres on layer and embedded convolutional layers. This netincludes dropout, data augmentation, ReLU nonlinearity and stochastic gradientdescent. The main difference of VGG [8] from AlexNet is application of filterswith 3x3 dimension. This feature allows to emulate larger receptive fields andwill used in Inception and ResNet [9] architectures afterwards. LSTM [2] (Longshort-term memory) - architecture of recurrent neural network (RNN), whichhave ability to learning of long-term dependencies. Instead of one layer, RNNhas four layers, that communicated by special way

ResNet [9] has less numbers of filters and lower complexity. A distinctivefeature of the architecture is the use of residual-connections, which enable tounite features from different layers and pass the result to the classifier. Residual-connections give the classifier access to feature information from different areasof the image. DenseNet is one of the ResNet [9] configurations, in which wasdeveloped new view on skip-connections: layers communicate not only with layersat the end of the block, but a large number of connections inside the unit areestablished. Distinctive feature of the SqueezyNet [10] is waiver flatten layersand normalization, that make net more lightweight and portable.

3 Examined Approach

Based on the fact that for today convolutional networks make it possible toget classifiers with accuracy of more than 99% for a large number of tasks andon different data sets, in this paper we examine the comparative analysis ofnetworks architectures, which can help to extract more features for resolvingemotion recognition problem.

3.1 Database

As training samples were used spaced open dataset SAVEE. Surrey Audio-VisualExpressed Emotion (SAVEE) database has been recorded as a pre-requisite for

34

the development of an automatic emotion recognition system. The database con-sists of recordings from 4 male actors in 7 different emotions, 480 British Englishutterances in total. The sentences were chosen from the standard TIMIT corpusand phonetically-balanced for each emotion. The data were recorded in a visualmedia lab with high quality audio-visual equipment, processed and labeled. Tocheck the quality of performance, the recordings were evaluated by 10 subjectsunder audio, visual and audio-visual conditions. Classification systems were builtusing standard features and classifiers for each of the audio, visual and audio-visual modalities, and speaker-independent recognition rates of 61%, 65% and84% achieved respectively.

3.2 Experiments

According to the previous research [7] on the first stage of experiment we im-plemented the preprocessing of dataset, which presented with set of .wav. Atfirst, audio signal was scaled with lowpass and highpass filters, with 30Hz and2700Hz borders respectively, because it is more suitable for human speech. In ad-dition,we used Voice Activity Detection to separate voice from background noise.After that, we applied Fast Fourier Transformation to obtain sound melspectro-grams. This set of images was used in the training of convolutional networks andtheir ensembles. We used PyTorch library to construct models, which aggregatedfeatures of few models. The main idea of the experiment is to research quality ofextracting features by CNNs in such complex task as emotion recognition. Alsowe made experiments with another way of preprocessing - MFCC [3] coefficientsand extract features from the time series with LSTM [2] network.

Table 1. Accuracy of LSTM network and MFCC.

n layers 1 MFCC 2 MFCC 3 MFCC 5 MFCC 10 MFCC 15 MFCC

2 layer 27.9 30.4 30.0 26.5 27.8 27.33 layer 32.1 32.5 34.2 32.8 33.1 31.85 layer 31.9 32.6 32.5 32.5 29.8 29.9

3.3 Results

The preliminary goal was to find the optimal network to extract emotion fromaudioflow to solve the problem of classification. Experiments with LSTM net-works and MFCC seem like they contains less information about intonationthan melspectrograms. Among the single models the best result was achieved byDenseNet121.

As regards combination of nets: ResNet18 and AlexNet show best accuracy.The results can be explained by the network architecture, they extract different

35

Fig. 1. DenseNet121 accuracy.Red line - train, green line - vali-dation

Fig. 2. DenseNet121 loss.Red line - train, green line - vali-dation

features and both can get valuable information about emotions. In addition,quality has been improved due to complementarity of features.

Fig. 3. AlexNet+ResNet18 accu-racy.Red line - train, green line - vali-dation

Fig. 4. AlexNet+ResNet18 loss.Red line - train, green line - vali-dation

The combination of ResNet and AlexNet networks extracts features of vari-ous kinds, ResNet uses Residual-connections, which involves a greater number offeatures by communicating layers of different levels and allows not to lose signif-icant information from layer to layer. Combining ResNet with a simple AlexNetnetwork we can get more different features and use strengths of both networks.

36

Table 2. Maximal accuracy got by training not-pretrained nets on 100 iterations.

Name of CNN Accuracy

AlexNet 36.905ResNet18 57.142DenseNet121 65.476ResNet18 + DenseNet121 60.714Alexnet + ResNet18 61.905Vgg19 + SqueezyNet1 0 44.048DenseNet121 + SqueezyNet1 0 58.333Vgg16 + DenseNet121 57.143

Most of the models that were run with pretrained ImageNet [5] weights gavethe worst result because the features that are extracted from this dataset poorlydescribe the important regularities of the spectrograms and the random initial-ization from the normal distribution gave the best result

Table 3. Maximal accuracy got by training pretrained nets on 100 iterations.

Results of fit pretrained nets

VGG16 + ResNet18 VGG16 and ResNet18 58.333DenseNet121 + SqueezyNet1 0 DenseNet121 and SqueezyNet1 0 59.524VGG19 + SqueezyNet1 0 VGG19 and SqueezyNet1 0 46.429VGG19 + SqueezyNet1 0 SqueezyNet1 0 55.952AlexNet + ResNet18 AlexNet + ResNet18 64.286AlexNet + ResNet18 AlexNet 57.143ResNet34 + DenseNet121 ResNet34 + DenseNet121 54.763ResNet50 ResNet50 44.048AlexNet AlexNet 36.905

4 Conclusions and directions for further work

The result of our research is a comparative analysis of networks with differentfeatures and architectures. In addition, the complexity of the dataset has in-creased relative to the previous work [7], as the RAVDESS dataset contains2 phrases. We have compared networks architectures for feature extraction foremotion classification. Also, we got the result that pretrained on ImageNet net-works achieve lower accuracy than networks initialised from normal distribu-tion. It can be explained by the fact that the networks are pretrained to extractunimportant features for emotion patterns. Besides that initialization and pre-configuration of networks play an important role. In the future, we plan to focuson intonation research and work on data collection and creation of a dataset,because it is necessary to create a representative and valid database with various

37

phrases examples. It will allow us to get more information from intonations andless information from phrases.

References

1. Choi K., Fazekas G., Sandler M.: Automatic tagging using deep convolutional neuralnetworks, arXiv preprint arXiv:1606.00298. 2016.

2. Hochreiter S., Schmidhuber J.: Long short-term memory, Neural computation. 1997.. 9. . 8. . 1735-1780.

3. Muda L., Begam M., Elamvazuthi I.: Voice recognition algorithms using mel fre-quency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques,arXiv preprint arXiv:1003.4083. 2010.

4. Huang G. et al.: Densely connected convolutional networks. Proceedings of the IEEEconference on computer vision and pattern recognition. 2017. . 4700-4708.

5. Deng J. et al.: Imagenet: A large-scale hierarchical image database. 2009 IEEEconference on computer vision and pattern recognition. Ieee, 2009. . 248-255.

6. Krizhevsky A., Sutskever I., Hinton G. E.: Imagenet classification with deep convo-lutional neural networks. Advances in neural information processing systems. 2012.. 1097-1105.

7. LeCun Y. et al.: LeNet-5, convolutional neural networks http://yann. lecun.com/exdb/lenet. 2015. . 20.

8. Simonyan K., Zisserman A.: Very deep convolutional networks for large-scale imagerecognition, arXiv preprint arXiv:1409.1556. 2014.

9. He, K., Zhang, X., Ren, S. and Sun, J.: Deep residual learning for image recognition.In Proceedings of the IEEE conference on computer vision and pattern recognition2016 (pp. 770-778).

10. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J. and Keutzer, K:SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and¡ 0.5 MB modelsize. arXiv preprint 2016 arXiv:1602.07360.

38

Using neural network models for classification of short text messages

M. Dli, O. Bulygina

National Research University “MPEI”, Smolensk, Russiae-mail: midli@imail.ru, baguzova_ov@mail.ru

Abstract. Today public authorities are actively introducing the technologies of electronic interaction with citizens and organizations. Constantly increasing volume of complaints determines the need to use special information systems for automated classification and distribution of incoming messages. The features of such messages (small size, lack of a clear structure, etc.) do not allow using traditional probabilistic-statistical approaches, and this fact leads to the need for application of mining methods. The article suggests the use of neural-network models (artificial neural networks and neuro-fuzzy classifiers) for rubrication of short text message, while their choice is determined by the rubric features.

1 Introduction

Total informatization of human activities leads to the development of computerized linguistics engaged in the automatic processing of textual information. One of the urgent tasks of analyzing such information is the development of methods for the classification of electronic unstructured text messages written in natural language. First of all, this is due to the need to process large volumes of electronic text messages received on the Internet resources of various organizations and institutions.

This problem is acute for public authorities, which are actively introducing the technologies of electronic interaction with citizens and organizations. The annual increase in the number of messages, received on their Internet portals and e-mails, leads to the need for use of automated analysis systems to promptly distribute these complaints among the various departments which will process them. In this case, the task of classifying electronic text messages is to distribute them into thematic rubrics that determine the activities of various departments.

The choice of classification method is determined by the specifics of the analyzed message, the number of rubrics and the features of their formation (first of all, the degree of variability of the rubric field thesaurus).

The distinctive characteristics of electronic text messages received by public authorities are their small size, lack of a clear structure, free presentation style, variety of types of the requests (proposals, applications, complaints, etc.) and the issues described in them.

These features impose certain restrictions on the application of traditional approaches to the text analysis. In this regard, it is advisable to use the methods of intellectual analysis of text data that allow processing unstructured message under conditions of variability of the rubric thesaurus.

The foregoing determines the relevance of the scientific task of developing methods for analyzing and classifying electronic unstructured text message taking into account the specifics of their content and usage in electronic public services system.

2 Related works

Currently, a large number of Russian and foreign publications are devoted to text classification using data mining methods.

Artificial neural networks are a powerful tool of machine learning that allow finding hidden patterns in message written in a natural language [3, 6, 10]. It is proposed to use several architectures of artificial neural networks to solve the problems of text classification [2, 14]. So, the convolutional [9, 11, 13, 19], recurrent [8], recursive [7, 17] networks and auto-encoders [15, 16] show good results.

Also, neuro-fuzzy classifiers [1, 4] can be used to classify text message. They allow solving multi classification problems under conditions of limited statistical information.

However, each mathematical method has special conditions of applicability (first of all, these are features of learning algorithms). It means that today there is no universal tool for solving the task of classifying texts.

In the article [5], the authors proposed to use several classification models depending on the text characteristics (size, degree of rubric thesaurus intersection, amount of accumulated statistical information) when developing information system for automatic analysis.

3 Features of apppying neural network models for the classification of short text message

In general, there are two types of classification of the text message: binary and multiclass.

Binary classification answers the question whether this mesage is interesting (the answer is “yes” or “no”). Logistic regression is usually used to implement this type of classification.

Multi classification refers the message to one (or several) class of a set consisting of three or more elements. This type of classification can be implemented in two ways:

1) Using Softmax function that calculates the fractional probability ofassigning a message to each class (the sum of the probabilities is equal to 1). It is applicable only in the case where the message relates to the one class. This function is often used in artificial neural networks.

40

2) Repeated application of binary classification, i.e. building a separateclassifier for each class of the set. For example, this approach can be implemented using a neuro-fuzzy classifier.At the same time, for the successful use of neural network models, it is necessary

to have sufficient statistical information for training.The relations between rubrics can be used as a criterion for choosing the method

for classification of short text message. So, if there is the rubric thesaurus intersection, it is advisable to use the neuro-fuzzy classifier, otherwise – the neural networks.

4 Classification model based on neuro-fuzzy classifier

In the process of classifying short text messages, there are situations when classes (rubrics) do not have clear boundaries or their sets intersect. In this case, the neuro-fuzzy classifier can be used.

In general, neuro-fuzzy classifier is a type of neural network that is the adaptive equivalent of a fuzzy inference model. The essence of this apparatus is to form a system of fuzzy rules (expert knowledge) representing the procedure of obtaining conclusions on a given set of assumptions [12]. In this apparatus, fuzzy inference algorithms are implemented as a neural network having heterogeneous layers of neurons.

However, the above features of the message, received by the Internet resources of public authorities, do not allow explicit use of this mathematical apparatus, i.e. there is a need for its modification.

In well-known models, using a neuro-fuzzy classifier, text message is represented as an array of binary values characterizing the presence or absence of words from the thesaurus for each rubric field. However, this approach is difficult to implement under conditions of thesaurus dynamism due to the need to rebuild the neural-fuzzy network and the model of message formalization when the rubrics change. To solve this problem, it is proposed to introduce the classification model in the form of a set of submodels [18].

The authors have proposed a neuro-fuzzy model, which allows analyzing short text messages based on their unified presentation. It includes the following submodels:

1. A submodel for preliminary analysis using a syntactic parser. It is intended forthe formation of a set of significant words for the text message.

2. A submodel for formalization using weight coefficients. It is intended todetermine the degree of belonging of syntactic groups to rubrics.

3. A set of submodels for assessing the belonging to the individual rubrics. Eachsubmodel is implemented as a neuro-fuzzy classifier.

4. А submodel for selecting the rubric that is most relevant to the analysed textmessage.

A detailed structure of the neuro-fuzzy model for assessing the belonging of short text message to the rubrics is shown in Figure 1.

41

11

21

n1

12

22

13

23

J3

32

2

Est(SD1,Rj)

vv11

vv12

vv1J

vv21

vv22vv2J

vv31

vv32

vv3J

vv 1vv3J

vv 2

(k)

Est(SD2,Rj)(k)

Est(SDn,Rj)(k)

n

Est(SDN,Rj)(k)

Rj

Figure 1. A detailed structure of the neuro-fuzzy model for assessing the belonging of short text message to the rubrics

The inputs of elements of the first layer of the neuro-fuzzy model receive the values of assessing the degree of correspondence of the words of the syntactic

characteristic n of message k to the rubric j – ( )( , )kn jEst SD R .

The elements of the second model layer implement the fuzzy activation functions for rules that evaluate the influence of the analyzed word on the rubric definition. They are term sets corresponding to the values: “weak”, “medium” and “high” effect. The model uses functions of triangular type.

The elements of the third model layer implement the calculation of the minimum function for all input values. The number of neurons of this layer is 3N, and the coefficients vv of neurons are adjusted during the training.

The fourth model layer consists of J elements that implement the maximum function.

As a result, the degree of the belonging of short text message to the rubrics j is formed at the output of the private neuro-fuzzy model.

42

5 The use of neural networks to classify short text message

The analysis of Russian and foreign publications on text classification using artificial neural networks revealed that convolutional and recurrent networks show good results at this sphere.

Convolutional networks are a type of feedforward neural networks when a signal travels sequentially along network neurons (from the first layer to the last). They were originally developed for image analysis. Good results in this area contributed to the research on their application for solving other classification problems, including for text messages.

In general, convolutional neural network is an alternation of convolution, subsampling and fully-connected (at the output) layers. All three types of layers can alternate in random order.

It is proposed to use this type of artificial neural networks to classify short text messages when rubrics do not intersect. In this case a sentence arrives at the network input. Each word in the sentence is determined by a vector (for example, it can use the word2vec algorithm to translate into a vector format). The output layer implements Softmax functions used for multi classification.

However, there are the situations when the order of the words in rubrics names and phrases defining their thesaurus is important for defining these thematic rubrics. So, the task of classifying sequences arises when it is necessary to consider the context of the phrase. The recurrent neural networks are one of the successful tools for solving such classification problem.

Recurrent networks are a type of feedback neural networks when the neurons use information from the previous layer and data about the state of these neurons in the previous pass. In this case the outputs in the intermediate steps are not used, and the last output of the neural network returns the predicted class. Similarly, it is proposed to use the Softmax function to exit the neural network.

6 Conclusion

The article proposes a new approach to the use of neural network algorithms to solve the problems of classification of electronic unstructured text message, received by the Internet resources of public authorities. The authors have proposed three situations of classification of short text messages that determine the choice of neural network algorithms:

1) convolutional neural networks can be used in the case of the unambiguousdefinition of thematic rubrics;

2) recurrent neural networks are applicable in the situation when the word orderand sentence context is important in determining the rubrics and significantwords;

3) neuro-fuzzy classifier should be used in the case of rubric thesaurusintersection.

43

7 Acknowledgment

The reported study was funded by RFBR according to the research project 18-01-00558.

References:1. Avdeenko, T., Makarova, E.: Acquisition of knowledge in the form of fuzzy rules for

cases classification. Lecture Notes in Computer Science. Data Mining and Big Data, vol. 10387, pp. 536-544 (2017).

2. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A Neural Probabilistic LanguageModel. JMLR 3, pp.1137-1155 (2003).

3. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Naturallanguage processing (almost) from scratch. JMLR 12, pp.2493–2537 (2011)/

4. Dli, M., Bulygina, O., Kozlov, P., Ross, G.: Developing the economic informationsystem for automated analysis of unstructured text documents. Journal of Applied Informatics, vol. 13, no. 5 (77), pp. 51-57 (2018).

5. Dli, M., Bulygina, O., Kozlov, P.: Development of multimethod approach to rubricationof unstructed electronic text documents in various conditions. Proceedings of the International Russian Automation Conference (RusAutoCon), Sochi (2018).

6. Dli, M., Bulygina, O., Kozlov, P.: Formation of the structure of the intellectual systemof analyzing and rubricating unstructured text information in different situations. Journal of Applied Informatics, vol. 13, no. 4 (76), pp. 111-123 (2018).

7. Iyyer, M., Enns, P., Boyd-Graber, J., Resnik, P.: Political Ideology Detection UsingRecursive Neural Networks. Proceedings of ACL 2014 (2014).

8. Kalchbrenner, N., Blunsom, P.: Recurrent convolutional neural networks for discoursecompositionality. Workshop on CVSC, pp. 119-126 (2013).

9. Kim, Y.: Convolutional neural networks for sentence classification. IEMNLP,September, pp. 1746 -1751 (2014).

10. Kozlov, P.: Automated analysis method of short unstructured text documents.Programmnye produkty i sistemy, no. 1, pp. 100-105 (2017).

11. Krizhevsky, A. Sutskever, I., Hinton, G.: Imagenet classification with deepconvolutional neural networks. NIPS, pp. 1106 -1114 (2012).

12. Kruglov, V., Dli, M., Golunov, R.: Fuzzy logic and artificial neural networks. Moscow:Nauka, Fizmatlit (2001).

13. LeCun, Y. Text understanding from scratch. Computer Science Department (2016).14. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed Representations

of Words and Phrases and their Compositionality. Proceedings of NIPS 2013 (2013).15. Socher, R., Huang, E.; Pennington, J., Ng, A., Manning, C.: Dynamic pooling and

unfolding recursive autoencoders for paraphrase detection. NIPS, vol. 24, pp. 801-809 (2011).16. Socher, R., Pennington, J.; Huang, E., Ng, A., Manning, C.: Semi-supervised recursive

autoencoders for predicting sentiment distributions. EMNLP, pp. 151-161 (2011). 17. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C., Ng, A., Potts, C.: Recursive

deep models for semantic compositionality over a sentiment treebank. EMNLP, pp. 1631–1642 (2013).

18. Tukaev, D., Bulygina, O., Kozlov, P., Morozov, A., Chernovalova, M.: Cascade neural-fuzzy model of analysis of short electronic unstructured text documents using expert information. ARPN Journal of Engineering and Applied Sciences, vol. 13, no. 21, pp. 8531-8536 (2018).

19. Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for textclassification. Advances in Neural Information Processing Systems, Febrary, pp. 649-657 (2015).

44

adfa, p. 1, 2011.

© Springer-Verlag Berlin Heidelberg 2011

Telepresence or video call? Which improves the way we

communicate?

Rivosoaniaina Alain Nimbol, Mahatody Thomas and Josvah Razafimandimby1

1Departamento de Ciencias de la Computación

Universidad de Alcalá (España)

luis.bengochea@uah.es

Abstract. Since the advent of cloud computing, collaborative work environments

have emerged as the dominant interaction style and most remote communication

systems are based on these principles. However, there is a considerable loss of

information and a tremendous amount of effort during communication. This ar-

ticle presents the design and evaluation of a telepresence robot compared to the

traditional system. Incidentally, we will offer a low-cost immersive telepresence

solution. Rather, our approach has been to offer a robotic immersive telepresence

system that is easy to implement and the cheapest.

Mots clé: telepresence; communication; virtual reality; development kit; HMD;

LeapMotion; Head Mount Display;

1 Introduction

When communicating face-to-face, a lot of information is encoded in our movements

(our gestures, body posture or head posture), which implies that 7% of human commu-

nications go through the words [1] and the 93% remaining are nonverbal. However,

there is a more than 90% loss of information and a colossal effort deployment during

communication, but socially richer robotic immersive telepresence could help over-

come these limitations. Telepresence refers to a set of technologies that allow users to

feel at a distance. The tele-robotics is a subfield of telepresence. The telepresence robots

are rapidly finding applications in areas ranging from offices and public spaces to space,

through the marine and submarine[2], telemedicine [3], agriculture, military environ-

ments and dangerous[4]. There are as many definitions as there is scope for

telepresence. The constraints it faces are also numerous, which has led to multiple def-

initions of telepresence [5][6][7].

2 Background

According to several studies on telepresence robots[8][9] [10], they are ideally ap-preciated by the users where it is placed. The more the robot recreates the shape of the human hand, the more the feeling of presence is important. The complexity of robotic

systems varies considerably, from simple axis with pliers to a fully robotized humanoid body. The characteristic that we will present here concerns only robotic telepresence.

2.1 Telepresence for communication

Several hypotheses exist to succeed a system of communication through robotic telepresence according to each specific field of use, since Marvin Minsky [5] in 1980 until today. Let's take the example of Sigurdur Orn Adalgeirsson[11], he proposed some hypothesis:

Copresence: People would feel greater co-presence when interacting with a hu-manoid-type robot that draws on human expression.

Psychological involvement: People would be more involved psychologicallywith their interlocutor when they interacted with an expressive telepresence ro-bot.

Confidence: People would trust their interlocutor more when interacting with anexpressive telerobot.

Engagement: People would feel more engaged with their interlocutor when in-teracting with an expressive telerobot.

Cooperation: people would cooperate better with their interlocutor when theyinteracted with an expressive telerobot.

Fun: People would appreciate their interaction more when interacting with anexpressive telerobot.

2.2 Ideal robotic telepresence system

The improvement of telepresence depends on a part of the evolution of the correspond-

ing technologies, more particularly to reach the perfect presence [6]. It is difficult to

define correctly, an ideal telepresence system even to handle subjective. The ideal ro-

botic telepresence system is what makes the system totally immersive and collaborative

that does not cost a fortune. Submission of the sense organs from the respective devices

increases the level of immersion (sight, hearing, touch, smell and taste is still experi-

mental). All this information does not help us to define the ideal telepresence system,

however, here are some of the assumptions we made about it:

Two-way communication,

Interactive control that is natural,

Transmission of information in real time,

A visual feedback system,

Hearing simulation increases immersion,

The return of effort gives the feeling of being present.We have identified more than 120 telepresence systems, we have grouped systems

of the same family, of the same producer, but with a difference of version as being a single system. Thus, we have presented 68 more relevant systems in Figure 1.

46

Fig. 1. Summary of the review of robotic telepresence systems

Legend:

• Second column and Fourth column:

o HMD (Head Mount Display)

o PC (Personal Computer)

o G (Glove, Gauntlet)

o SP (Smartphone)

o T (Tablet)

o J (Joystick)

o GP (Game Pad)

o - (Dash means uncertain or nil)

• Fifth column:

o P (Personal)

o E (Experimental)

o C (Commercialize)

• The last column:

o VR (Virtual Reality)

o AR (Augmented Reality)

o AV (Augmented Virtuality)

o MR (Mixed Reality)

o Ai (Interactive Application)

o WA (Web Application)

• Extensive literature[7][12] finds that the quality of audio communication can be a

major obstacle to collaboration and fluid interaction[13]. We can see that all these

systems use sound and video.

• Of telepresence systems, 84% do not use instant messaging. Textual communication

is not considered VR[12] because it does not improve immersion, but rather de-

creases.

• Nonverbal communication[14][15][16] plays an important role in coordinating the

actions of teammates for collaborative activities. Of the 68 tools cited in Figure 1,

only 12 can interact to make a decision or to reproduce a gesture of the user.

• There are different other messages that are not in the above-mentioned categories, for

example, user emotion (smile, acquiesce, etc.), collaborative work data (report, sta-

tistics, other documents, etc.). Most collaborative systems identified, 61.76% more

exactly do not allow the transmission of this kind of message.

• In our list, we can see that the majority uses the PC and Mac, precisely 38.24% of the

systems. Then, the tablet, smartphone or PC occupy 16.18%. Some systems use spe-

cial tools, such as LEGOS, KINECT or iPad.

• Robotic telepresence affects different areas of application. Some tools of our census

do not have clear membership that we have chosen to generalize them in the field of

social communication. In the nuclear field which constitutes 3.64%, and the under-

water domain 4.55%, the space domain 11.82%, the field of education occupies

10.91%, the field of security occupies 5, 45%, the medical field occupies 16.36%, the

field of assistance constitutes 8.18%, the field of exploration constitutes 13.64% and

the field of social communication occupies the vast majority of which the rest 25.45%.

48

• For navigation systems of a telepresence robot, the HMI plays an important role. By

guiding these kinds of robots, one is preoccupied with control, and one loses the feel-

ing of presence[12], because the immersion decreases. Conventional navigation de-

vices (joystick, joystick, etc.) are the tools to guide robots, and 20.59% of the above

telepresence robots use them. These tools are replaced by tactile devices (tablet,

smartphone, touchscreen, etc.), as 25% use these interactive tools. Most use a com-

puter or Mac, 48.53% to be more precise. But with the advent of virtual reality tools,

we can see that the remaining 5.88% of the telepresence robots that we have identified

are guided by the HMD.

• The telepresence study is gaining more and more ground in research, but also in the

productive industries. As a result, 60.29% of robotic telepresence systems are com-

mercialized. The other 25% made the subject of studies in research laboratories. The

remaining 14.71% are personal ambitions.

• Of these systems, only 17.39% use virtual and augmented reality, and 5.80% operate

in a mixed reality environment. The other systems use interactive applications on one

of the standard devices, the 40.58% (Laptop, tablet, smartphone, etc.) and the 23.19%

use web browsers to access the applications.

Telepresence is primarily a communication tool that aims to optimize the perfor-

mance of standard information systems. In [17], we made a classification of the mes-

sages sent according to the tool. Let's add to the message types the mentioned criteria

rather to better subjectively define the ideal telepresence system.

Table 1. The few assumptions combined with the criteria of a communication system to have an

ideal telepresence system

Criteria Telepresence Standard Téléprésence Ideal

Son X X

Speech X X

Video X X

Gesture - X

Text - -

Natural Interactivity - X

Stress feedback - X

Bidirectional communication X X

Real Time Communication X X

There are other criteria that we have omitted from this ranking, namely, the area of

application that in itself directs the design and realization of an adequate system. The

budget to acquire a robotic telepresence solution as needed. It is clear that all these

shortcomings prevent us from defining an ideal system.

3 Method

In this section, we will present the design part as well as the description of the system

used. To do so, we will see the hardware configuration of the project that defines the

49

components and devices input and output the system operate for the navigation system

and interaction. With telepresence, the user can have the opportunity to act on the re-

mote site. In this case, the position of the user as well as his movements, actions, or

words can be perceived, transmitted and duplicated to the desired destination to put this

effect into action. As a result, the information can travel in both directions between the

user and the remote location.

This study aims to determine how a robotic telepresence system can be used as a

communication system. This difficult question requires dealing with interaction opti-

mization. Thus, we decided to design a robotic platform for mobile videophone of a

robotic telepresence system. In no case, this study does not seek to make improvements

in the field of robotics which is a fairly advanced discipline. Our robotic platform is

very rudimentary but sufficient to replace the telepresence robot that we lack and that

is expensive.

We designed a telepresence robot using the basic digital electronics kit, the Rasp-

berry Pi 3 pair with a MEGA 2560 Arduino microcontroller. We used a Raspberry Pi 3

based kit on which 1 Pi NoIR cameras are plugged. The MJPG-Streamer Stream Pro-

cessing API is installed and allows real-time video transmission. On this Nano com-

puter, a USB microphone and a speaker on the 3.5mm jack is connected. We also used

an audio stream management API, the G-Streamer for Raspbian. We used an Arduino

board to drive all the joints of the robot, like the SG90 servomotors mounted on Pan

TILT for the reproduction of the movements of the head to turn vertically and horizon-

tally. A servomotor from SG92R guides the clamp in front of the robot, this clamp

reproduces the interaction of the hand. An ultrasonic sensor is mounted on a servomotor

for the automatic navigation system. All of these components are mounted on a sepa-

rately powered two-wheel chassis, allowing it to rotate in place.

Fig. 2. The prototype telepresence robot used in this work

50

We used the HMD OSVR [18] for immersion and to most faithfully reproduce the

interaction of the user's head towards the robot. Navigation with the robot is done with

the LeapMotion motion sensor [19] and the right hand through a robot control interface.

To advance, we open our right hand and move forward, the same goes back and back.

The driver determines the speed by interacting with the control interface. The major

disadvantage of this type of interface was that the user had to keep his hand in the

interaction field of the LeapMotion so that he does not lose control. If the user was

facing in another direction, it was very difficult and confusing to steer the robot. We

have a preliminary description in the previous work [17].

Fig. 3. On the left (a) the user with the interface using the PC, HMD and LeapMotion..

The menu bar in Figure 3 on the right shows the features available during the com-

munication. In the lower left corner, the left and right hand icons, when present in the

LeapMotion controller field. The icons at the bottom right indicate the presence of the

OSVR headset and the status of the robot (Off, Forward, Reverse, Left or Right, etc.).

4 Experimentation and Result

In order to be convinced that the improvements brought by telepresence are well perceived by the users, it was decided to make a small series of validation tests. The main objective of this experiment is to validate the telepresence solution that we pro-posed. "Any evaluation consists of comparing a model of the object being evaluated with a reference model for making conclusions." [20]. In order to evaluate the system as a

51

communication tool against a video call, both communication methods were evaluated via a user study. The video communication tools used in this evaluation were the stand-ard tools that many use Skype, Messenger, Whatsup or Hangout.

A qualitative test was conducted with 13 participants, 7 men and 6 women. The peo-ple who did the test were 17 to 35 years old. Half of the test group had experience con-trolling remote virtual characters in video games, while the other half had little or no experience with this type of action. Each test took between 15-20 minutes task.

4.1 Evaluation method

The evaluation was conducted in two rooms and required two participants per round. This included 2 iterations: one to evaluate telepresence communication and one to eval-uate video communication. Each iteration consisted in solving a sudoku problem where the remote user saw the solution and had to communicate it to the other person who was physically at the remote site, presented in figure 8. Once the puzzle was resolved, the participants answered a questionnaire about how they perceived communication. The time required to solve the problem was measured with a stopwatch. It was also noted if the sudoku was correctly placed or if there were errors.

Fig. 4. The participant on the site saying where the robot is with the sudoku the hand.

Each iteration consisted of a brief explanation of the piloting interface, after which each person in charge of the test was asked to perform a task with the robot, such as moving an object from point A to point B. And when the first task has been completed, the test taker including the user has been informed of the next task while still wearing the HMD. For the first task, the test taker was asked to follow the path described in Figure 4 on the right.

52

Fig. 5. the map moving the robot in the first task of driving the robot from the starting position

(S) to the point (A), then to point (B).

4.2 Result of telepresence for communication

For the evaluation of telepresence, there is no concrete evaluation system. We have been led to create some evaluation criteria inspired by the several methods [12][21][6]. Ralph Schroeder [12] has studied the field of social interactions in a virtual environment. He argued that to study communication in shared virtual environments, it will be neces-sary to combine different perspectives, including social psychology, sociological analy-sis of interaction and approaches to communication studies for different media. How-ever, this study does not allow us to have a way of measuring communication.

For his part, Martin Hassell and his colleagues [21] proved the flexibility of these variables by using them in a similar field. He studied the effect of seeing a mirror version of himself in video communication. They built an experiment in which small groups of people performed a group decision exercise using video communication. Half of the groups were able to see their own video stream as well as the videos of the other mem-bers of the group, the other half saw only the video stream of the other members of the group. According to Thomas B. Sheridan [6] in 1992, in order to better define presence, we need the three main determinants:

The extent of sensory information,

Control of the relationship of the sensors with the environment,

The ability to modify the physical environment.

By taking inspiration from all these theories mentioned above, we have defined 5 criteria to evaluate our communication system. Indeed, for the experiment we conducted, telepresence for communication was evaluated according to 5 criteria: notably its effi-ciency, user satisfaction with the process, satisfaction with the solution, co-presence that defines the feeling mutual presence of interlocutors and the cognitive load of the user. Efficiency has been measured over time and accuracy. The other values were measured by a questionnaire to which participants responded. This makes it possible to analyze these criteria separately for the user and the interlocutor.

53

In order to determine an overall result of the communication tools, we asked questions for each criterion and the responses of the user and his interlocutor were combined and illustrated in Table 2.

Table 2. Combined outcome of the telepresence experience for communication

Criteria Telepresence A/V Video call

Efficiency 70,43% 40,87%

Satisfaction with the process 67,83% 64,78%

Satisfaction with the solution 58,26% 62,17%

Copresence 75,36% 40,58%

Cognitive load 75,65% 37,39%

This result shows that telepresence communication was significantly more efficient than usual standard video communication. The satisfaction of the solution was quite sim-ilar between the two methods of communication. The experience of telepresence had less cognitive load on users. Experience has also shown a strong increase in perceived co-presence compared to video communication.

However, video communication is always more satisfying for the process. Although the study did not involve remote communication, these communication measures are still relevant because they allow different methods of communication to be compared.

Fig. 6. Combined outcome of telepresence experience for communication

5 DISCUSSION

Telepresence or Video Communication? That is the question. To answer this ques-

tion, we performed tests, first to evaluate our telepresence system and then to compare

similar systems. Here are some relevant points that make up these tests:

Evaluation criteria: For video communication versus Telepresence, evalua-

tion criteria for measurement methods have been developed in different ways.

54

Tasks during evaluation: Since the test consisted of evaluating both

telepresence and video communication was created so that the user did not

know the solution beforehand when performing the communication method

next.

Telepresence wins: we can conclude that telepresence was clearly winning in

the comparison.

Two-way video: We tested two functional prototypes, version 1 or video com-

munication is only available to the local user and colleagues in the remote site

only see the robot. Some participants felt that video communication was faster

and more flexible when it came to moving. Another advantage of video com-

munication is that the interlocutor can see the face of the inhabitant.

Satisfactions: One of the interesting conclusions about user participation, alt-

hough satisfaction with the solution has visually better values than the tradi-

tional video communication method, participants seem to prefer slightly the

process with telepresence.

User and VR: The majority of users in the local site prefer communication

with video telepresence.

The telepresence robot: the limitation of the movements with the robot of

telepresence are part of the reasons of their choices. These users claim that a

human is much more apt when it comes to moving objects relative to a robot.

6 Conclusion & Limitation

During these tests, the majority of participants are accustomed to video communica-tions, so it can be said that familiarity with video communication with the test subjects could have influenced the outcome. It is likely that telepresence communications in the coming years will be in similar conditions and should corollary be ready to meet this challenge. To summarize, the experience with telepresence was the best communication tool for participants, both with respect to measurement criteria and the personal opinions of users. As a result of the study, one of the attempts to make the steering interface more flexible was to add two wheels with a steering arm to improve the way the robot would behave.

7 References

[1] Mehrabian, Albert. Nonverbal communication. . Routledge, 2017.

[2] Otmane, Samir. Téléopération, télérobotique et Internet: Techniques & applica-

tions. Université d'Evry Val d'Essonne CNRS-FR (2010) 2873.

[3] Satava, Richard M. Virtual reality and telepresence for military medicine. Annals

of the Academy of Medicine, Singapore (1997) 26: 118-120.

[4] Lawson, Wallace E;Bekele, Esube;Sullivan, Keith. Finding Anomalies with

Generative Adversarial Networks for a Patrolbot. In CVPR Workshops. 2017.

[5] Minsky, Marvin. Telepresence. (1980).

[6] Sheridan, Thomas B. Musings on telepresence and virtual presence. Presence:

Teleoperators & Virtual Environments (1992) 1: 120-126.

55

[7] Beigl, Michael;Gellersen, Hans-Werner. Ambient telepresence. In Proceedings

of the Workshop on Changing Places. 1999.

[8] Zanotto, Damiano;Rosati, Giulio;Minto, Simone;Rossi, Aldo. Sophia-3: A semi-

adaptive cable-driven rehabilitation device with a tilting working plane. IEEE

Transactions on Robotics (2014) 30: 974-979.

[9] Siciliano, Bruno. Robot control. (2019).

[10] Bryson, Steve. Virtual reality in scientific visualization. Communications of the

ACM (1996) 39: 62-71.

[11] Ađalgeirsson, Sigurđur Örn. MeBot: A robotic platform for socially embodied

telepresence. .2009.

[12] Schroeder, Ralph. Social interaction in virtual environments: Key issues, com-

mon themes, and a framework for research. In The social life of avatars. . 2002.

pp. 1-18.

[13] Coad, Peter. Object-oriented patterns. Communications of the ACM (1992) 35:

152-159.

[14] Burgoon, Judee K;Guerrero, Laura K;Floyd, Kory. Nonverbal communication. .

Routledge, 2016.

[15] Knapp, Mark L;Hall, Judith A;Horgan, Terrence G. Nonverbal communication in

human interaction. . Cengage Learning, 2013.

[16] Breazeal, Cynthia;Kidd, Cory D;Thomaz, Andrea Lockerd;Hoffman, Guy;Berlin,

Matt. Effects of nonverbal communication on efficiency and robustness in hu-

man-robot teamwork. In Intelligent Robots and Systems, 2005. (IROS 2005).

2005 IEEE/RSJ International Conference on. 2005.

[17] Nimbol, Rivosoaniaina Alain;Thomas, Mahatody;Razafimandimby, Josvah Paul

. New type of communication: Immersive telepresence with OSVR and Leap-

Motion. In International Academic Multi-Disciplinary Conference (IAMDC -

Nov' 2018). 2018.

[18] Quin, Caramel. The teardown: Razer OSVR HDK2 virtual reality headset. Engi-

neering & Technology (2016) 11: 80-81.

[19] Zhang, Qixiang;Deng, Fang. Dynamic Gesture Recognition based on LeapMo-

tion and HMM-CART Model. In Journal of Physics: Conference Series. 2017.

[20] Senach, Bernard. Evaluation ergonomique des interfaces homme-machine:

une revue de la littérature. .1990.

[21] Hassell, Martin D;Cotton, John L. Some things are better left unseen: Toward

more effective communication and team performance in video-mediated interac-

tions. Computers in Human Behavior (2017) 73: 200-208.

56

The model of special computer interface for learning

adult students

I.B.Bashmakova

Saint-Petersburg State University of Architecture and Civil Engineering, Saint-Petersburg,

Russia

e-mail: delovoi2004@mail.ru

Abstract. This investigation is carried out to create a model of special

computer interface interface for learning adult students. Author suggests taking

into account such factors as user’s experience, individual phychological

characteristics in order to create most suitable scripts for computer supporting

learning and choice motivational game cases. The research helps to find the

best interface for adult students.

1 Introduction

An educational scenario of learning software has to provide the deep

knowledge of subject. Theory of human-computer interface studies the interactions

and the relationships between humans and computers. At first human-computer

interface was focused on interfaces using windows, icons and so on. As soon as

interface problems were investigated, the researchers started to concern with shared

understanding and explanations of human actions. The new essential challenges are

improving the way people use computers to work. User modeling was derived from

the need to provide support for human-computer collaboration as shared goals and

learning.

The importance of user modeling was evidenced by their increasing

influence in the design of software applications.

Elderly students study in order to decide their social problems. The main

difficulty of their training is high-tempo environment. To solve a given math task,

users sometimes use paper-based records to support performance. These records are

operating documents that serve as cognitive aids, and have been shown to improve

quality of knowledge by formalizing math algorithms. Conspectus is not new mean to

train practice and have been widely in education process.

However, there are important differences between paper conspectus and

interactive computer aids. It would be very effective to use different information

sources and canals of perception in train of elderly students.

To realize new challenges and opportunities for education we have to change

our point of view on active using electronic aids with intuitive understandable and

interactive interface. Software developers must address these high-tempo, distributed

demands.

Elderly users of educational programs have to learn dealing with team-based

issues of coordination and communication [8, 9, 10, 14, 15].

How we must organize user interface in order to effectively learn older students

with the help of contemporary technologies in short time? This article gives answer to

this question.

2 Related works

The problem of rational use of computers in the educational process is well known. Table 1 consists of the most important articles of the topic under consideration. A complete list of works devoted to various aspects of our topic, presented by different authors in [1-17].

Table 1

Name of article Author(s) Main statements of

article

Mental Models in Human-

Computer Interaction. Research

Issues About What the User of

Software Knows

John M. Carroll and Judith

Reitman Olson(editors),

“At present, there is no

satisfactory way of

describing what the user

knows. There way to

characterize the

differences among users

of various systems as

they go through the

process of developing an

awareness and

understanding of how

the system works or how

a given task is to be

performed”

Designing Inclusive Interfaces

Through User Modeling and

Simulation

Pradipta Biswas, Peter

Robinson & Patrick Langdon

“This article

presents a simulator that

cans reflect problems

faced by elderly and

disabled users while

they use computer,

television, and similar

electronic devices. The

simulator embodies both

the internal state of an

application and the

perceptual, cognitive,

and motor processes of

its user”

User Interaction Modeling and

Profile Extraction in Interactive

Systems: A Groupware

Application Case Study

Cristina Tîrnauca, Rafael

Duque and José L. Montaña

“This work constitutes a

methodological

contribution capable of

identifying the context

of use in which users

58

Name of article Author(s) Main statements of

article

perform interactions

with a groupware

application

(synchronous or

asynchronous) and

provides, using machine

learning techniques,

generative models of

how users behave”

The use of a game-based learning

platform to teach mathematical

statistics

Urazaeva L. “Didactical games

always gained attention

as a technique to

motivate students and

improve learning”.

Effective solutions to improving

mathematics and science

education

Urazaeva L. The effective solutions

to improving

mathematical education

are “based on the

development of

innovative computer

educational programs

which are able to

identify the personal

features of students”

On the base of study papers it is possible to define main internal and external

factors affecting the achievements in mathematics of elderly students (Figure 1)

Figure 1. Main internal and external factors affecting the achievements in mathematics of

elderly students

Motivation is very important for learning mathematics. Special computer game

environment and strong system of mathematical case-studies play an important role in

improving the math performance. Currently, most researches related to the impact of

using game are conducted for learning integral equations [9], mathematical statistics

[15]. There is a lack of related research in the context of learning math old-aged

59

students. The usage of e-learning systems has made educational material widely

accessible to elderly students. There is thus a great need to accommodate learning

process for individual differences [10, 14]. Therefore, research topic is necessary for

conducting. This research explores important aspects of the organization of computer

scenario and interface in learning software and their impacts on the student's

knowledge. The results could be a base for creation learning software for adult

students.

3 Model description

Author suggests using the F-model of interface (F=F(<A, B>, <C,D>), A=<A1(t),A2(t)>-levels of user experience and state of environment,, B=<B1(t),B2(t),..,Bn(t))-individual phychological characteristics, C=<C1(t),C2(t),..,Cm(t)>-scripts for computer supporting learning, D=<D1(t),D2(t),..,Dk(t)>motivational game case-studies. The F-model provides creation of rational interface of learning computer program.

The logical part of the rule’s system describes by function F= F(<A, B>, <C,D>)(Fugure 2, Figure 3). The linguistic part includes a system of questions and answers.

Figure 2 The logical part of the rule’s system

Figure 3 Special interface choicing

60

Computer training in a game form stimulates collaboration in the educational process. The active use of educational materials allows you to quickly apply knowledge in solving professional problems by the mathematical methods. The activity diagram is shown in Fugure 4.

.

Figure 4 Activity Diagam

First, it is necessary to fill the database tables with a large amount of test data in

order to take into account all cases and satisfy all the requirements of future users

(Fugure 5).

Then, developers should formulate rules for building an individual scenario and

the choice of learning examples for each student.

Figure 5. Conceptual software architecture

Author suggests using the case-study to learn probability theory. This case-study is named “Newsboy problem”. This problem is described in Inventory theory.

61

A newsboy has to purchase a quantity of newspapers for the day's sale. The purchase cost of the papers is $0.20 and they are sold to customers for a price of $0.35. Papers unsold at the end of the day are returned to the publisher for $0.05. The boy estimates that the mean demand during the day is 250 and the standard deviation is 50. Normal distribution is assumed. How many papers should he purchase?

Figure 6. Example of visualization (picture of newsboy is free for commercial use,

https://pixy.org/323746/)

Usage of software prototype demonstrated the increasing of the student’s

motivation.

26 students from two groups consist of 34 persons (part-time form of education)

were able to construct their own professional case on the base of “Newsboy problem”

and solve it using with real data (76.47%)

5 Conclusion

This article presents a systematic approach to solving the problem of training older students. The author offers a description of the software model and analyzes the use of the prototype. The program creates a user-friendly interface based on a combination of factors which impact on mathematical achievements in computer learning.The proposed tools confirmed the practical usefulness of this model for distance learning.

62

References:

[1]. Biswas P., Robinson P. & Langdon P. Designing Inclusive Interfaces Through User

Modeling and Simulation. International Journal of Human Computer Interaction, vol.

28, #1, pp. 1-33(2012)

[2]. Burke R., Hybrid Recommender Systems: Survey and Experiments. User Modeling

and User-Adapted Interaction, vol. 12, pp. 331-370(2002).

[3]. Carroll J. M. and Olson J. R. (editors), Mental Models in Human-Computer

Interaction Research Issues About What the User of Software Knows. National

academy press Washington, D.C. 54.p(1987)

[4]. Castellar E.N., Loo J.V., Szmalec A. and de Marez L., Improving arithmetic skills

through gameplay: Assessment of the effectiveness of an educational game in terms

of cognitive and affective learning outcomes, Information Sciences, vol. 264, pp. 19-

31(2014)

[5]. Erhel S. and Jamet E., Improving instructions in educational computer games:

Exploring the relations between goal specificity, flow experience and learning

outcomes. Computers in Human Behavior, vol. 91, pp. 106-114(2019)

[6]. Furió D., González-Gancedo S., Juan M.-C., Seguí I., Rando N. Evaluation

of learning outcomes using an educational iPhone game vs. traditional game,

Computers & Education, vol. 64, pp. 1-23(2013)

[7]. Hands G. L. and Stepp C. E. Effect of Age on Human–Computer Interface Control

Via Neck Electromyography Interacting with Computers, vol. 28, #1, pp.47-

54(2016)

[8]. Ke F., Computer-game-based tutoring of mathematics. Computers & Education, vol.

60, #1, , Pages 448-457(2013)

[9]. Keller A. and Ken Dahm K., Integral equations and machine learning, Mathematics

and Computers in Simulation, vol. 161, pp. 2-12(2019)

[10]. Kuurstra J., Individual differences in Human-Computer Interaction: A review of

empirical studies. Master’s Thesis in Psychology University of Twente Faculty of

Behavioural, Management and Social sciences, 57 p.(2015)

[11]. Patricia S. Moyer-Packenham, Christina W. Lommatsch, Kristy Litster, Jill Ashby,

Kerry Jordan, How design features in digital math games support learning and

mathematics connections,Computers in Human Behavior, vol. 91, pp. 316-332(2019)

[12]. Penner, D. E. Cognition, Computers, and Synthetic Science: Building Knowledge

and Meaning through Modeling. Review of Research in Education, vol.25: 1-

35(2000)

[13]. Pons-Lelardeux C., Galaup M., Segonds F. and Lagarrigue P., Didactic Study of

a Learning Game to Teach Mechanical Engineering, Procedia Engineering, vol.

132, pp. 242-250(2015)

[14]. Urazaeva L. Effective solutions to improving mathematics and science

education. INTED2018 Proceedings, pp. 2868-2873(2018).

[15]. Urazaeva L. The use of a game-based learning platform to teach mathematical

statistics. INTED2018 Proceedings, pp. 673-678. (2018)

[16]. Vitale J.M., Swart M.I. and Black J.B., Integrating intuitive and novel grounded

concepts in a dynamic geometry learning environment. Computers & Education, vol.

72, pp. 231-248(2014)

[17]. Wang K. and Sun WCh., Meta-modeling game for deriving theory-consistent,

microstructure-based traction–separation laws via deep reinforcement learning.

Computer Methods in Applied Mechanics and Engineering, vol. 346, pp. 216-

241(2019)

63

Method of Increasing the Accuracy of Measuring Signals

with the Use of Combined Test Algorithms

Mehdiyeva A.M., Mehdizade E.K. Azerbaijan State Oil and Industry University

almazmehdiyeva@yahoo.com

Abstract. The combined test algorithms increasing the accuracy of measurement by using the

simple additive and multiplicative tests have been developed and this enables to determine the

measured values of non-electrical quantities by the results of additional test measurements used

for identification of nonlinear transformation functions of information measurement systems.

Keywords: test method, increasing accuracy of measurement, nonlinearity of transformation

function, metrological characteristics, transformation function

1. Introduction

The main problem in designing contemporary measuring systems is the increase

of quality indices of their metrological characteristics (MC). It is known that this is

performed by the following two ways: making renewal in the constructions of initial

transformers-technological method, or by not intervening in their construction by the

algorithmic-test method [1, p.7]. In the present paper, by using the second method, at

the expense of extra information obtained by performing additional measuring

operations, special test equations are set up and the joint solution of these equations

increases the accuracy of measurement of measuring systems.

The goal of the paper is to decrease the N number test equations as far as possible,

to minimize the order of each equation and to determine the coefficients of

polynomials with high accuracy by using the algorithmic-test methods. For that, the

capabilities of the algorithmic-test method are analyzed, and possibility of making

combined simple test equations of different kinds was investigated.

2. Problem statement

It is known that in the general form, the transformation function of the measuring

system, its mathematical model of n order is written in the form of the following

polynomial [2, p.61-70]:

=

=n

i

i

ii ,xay0

(1)

where: x is the measured value contained in the input of MS; iy is the output value

of MS; −ia are the coefficients of the transformation function (TF) of MS and their

real values are defined by the test algorithms increasing the accuracy of measurement

at each periodical measuring. Therefore, a problem on definition of ia quantities of

TF with high accuracy is stated and change of the values of these coefficients with

respect to time ( ) .n,...j,xAj 1= is investigated. Changes because of external factors

and correlation dependences of these changes are determined, the errors of the results

of measurements and their constituents are studied and estimated. As a result, a

generalized structure of information-measurement system (IMS) is given.

3. Problem solution

Let us consider the elaboration of the algorithm for increasing the accuracy of

measurement based on simple additive tests. As by measuring with tests the TF of

initial MS depends on definition of n order unknown quantities n, a...,a,x 1

with high

accuracy, the system of equations consisting of )n( 1+ number equations is composed

[1]:

(2)

The expression (2) is a system of linear equations with respect to quantities

n, a...,a1 and is solved by the Cramer law. Writing the obtained values of the quantity

ia in (1), we get the following expression for the main test equation:

From the known values of n and from the set of tests determined from (2) we can

get the processing algorithm of MR. But as the TF of initial MS is nonlinear,

depending on the number of tests used in IMS and their quality ratio (ratio of the

number of additive tests to the number of multiplicative tests and soon) the obtained

algorithm makes difficult the determination of the sought-for quantity x from this or

other reason. Therefore, one of the main properties of optimality of the tests set used

simultaneously for identification of nonlinear TF in IMS is that the order of the main

test equation obtained at their realization is minimum. At the same time, the use of

tests in this or other form is connected with the accuracy received by θ additive and

k multiplicative constant components that they create. This feature is explained by

the fact that the accuracy of MN obtained by the tested IMS in the first turn is

determined by the accuracy of tests shaped in the system. In this connection we

should note that the simple additive test realized by the given accuracy in

comparatively simple measurement both of electric and nonelectric quantities is in the

form . Here θ is an additive constant component and has the same physical

nature with the quantity x . From the carried out investigations we get that any initial

measuring sensors designed on the basis of up-to-date technologies have no ideal

linear MC and the mathematical model of their TF in majority of cases is in the form

of a square, and very rarely in the form of a cubic polynomial. Even the curves in the

[ ]

( )[ ]

( )[ ]

=

=

=

=

−

=

−

=

=

−

=

−

1

1

1

1

22

1

1

11

1

1

...................................

)(

in

i

nin

in

i

i

n

i

i

i

n

i

i

i

xAay

xAay

xAay

xayo

θ±x

65

complicated form may be reduced to this form using the algorithmic-structural

approximation method [3, pp.32-39].

The measuring process consists of four stages. In the first two stages the quantities

x and - x in IS1 and IS2, in the next two stages the quantities θ+x and )x( θ+−

are measured. The processing algorithm of measurement result realized with respect

to measuring quantity x will be in the following form:

θ)yy()yy(

)yy(x

2143

21

−−

−

∆−∆

∆= . (3)

The present result has been obtained from the condition of identity of MS1 and

MS2. But in reality only such IMS-s possess the indicated properties, that the IT-s

contained in their structure have differentiation property [4, pp.65-68].

The mentioned factors rather restrict the using possibilities of the considered

method. But by the method suggested by us we can simplify the solution of the

problem by piecewise approximation [5]. It should be noted that only by using the

additive tests the differential scheme of the initial MS is not required.

In some cases, in MS, the joint use of the test methods for increasing the accuracy

of measurements, based on realization of additive test and the measurement method

based on inverse-transformation (IT) is possible. This time, the influence of the error

of IT on MR is excluded. The measurement process is constructed by the following

algorithmic succession. The measurement quantity x is transformed into the output

quantity 1y :

1a21 )(2

aaxy N +∆+=

Here x2a∆ and 1a are multiplicative and additive errors of MS1 and MS2,

respectively. The output quantity 1y in IT with the β transformation quantity is

transformed to the homogeneous quantity 1x : )(yx N ββ ∆+= 11 Here β∆ is the

current error of IT. The quantity 1x is transformed into 2y in MS2:

1a212 2a)]()(a[yy NNN +∆+∆+∆+= ββ ββ

Then the additive test 11 θ+y get the signal 3y :

)]()(a[yy NN ββ ββθ ∆+∆+∆++=2a12123

.

At the next step the additive test21111 θθθβ β ++∆++ )y()y(N adjoins the input

of MS2 and the result of the transformation becomes: )a(yy N 2a2234 ∆++= θ .

In the calculating device, by processing the results of measurements

4321 y,y,y,y with respect to the measuring quality x we get the following

expression:

21

341

1232

34

212θ

θ

θθ y

)yy(

yy

yy

yyx

−

−−+

−

−= . (4)

66

The realization of the algorithm (5) enables to decrease simultaneously the

additive and multiplicative components of the resulting measurement error using one

additive test. But this method has some restrictions in its practical realization:

- the identity of MS (MS1 and MS2) is required;

- when initial MS possesses nonlinear TF, the use tests make rather difficult

the processing algorithm. But, as it was mentioned above, by joint use of

approximation by parts and differential measurement methods, these restrictions are

eliminated and as a result we get a perfect TIMS.

As an another approach, we can consider practical realization of the method for

increasing the accuracy of measurement by using the additional measurements of

additive tests. This time, taking into account the restrictions and requirements

imposed in MS, we can note that if it is possible to form multiplicative test in the

system, then the test algorithm based on realization of the set of simple additive and

multiplicative tests would be more effective and universal.

4. The combined method increasing the accuracy of measurement

The test algorithms increasing the accuracy of measurement by the combined test

method usually should consist of the set of simple additive and multiplicative tests [1,

pp. 32]. By using the joint use of the test equations, the efficiency of accuracy of

measurement will express in the system the set of this or other kind of tests:

,NMA** =+

where *A is the amount of additive tests in the system, *

M is the amount of the

multiplicative tests, N is the amount of the quantities ia of the transformation

function (TF) of the measurement system. It becomes clear from the investigations

that the use of ( )1−= nA* test from one kind, *M =1 number of test equations from

another kind is considered as minimal limit and the identification of nonlinearity of

the transformation function of MS is obtained. But using separately the additive and

multiplicative tests, the identity of the main test equations is disturbed for the general

case. By applying a simple additive test in the form )n,.....,в(x)x(Aвв

1=+= θ the

main test equations take the form:

[ ]

[ ]=

≠≤≤

≠≤≤

≠

−Π

Π

=n

j

j,nв

в

jв,nв

в

j

jв

yy1

1

1

0

l

lθθ

θ. (5)

By applying a simple multiplicative test in the form ( )n,....,gxKg 1= , the

main test equations take the form:

[ ].

KK

K

yy

jg

ng ,j

g

jg,ng

gn

j

j

≠

≤≤ ≠

≠≤≤

=

−Π

−Π

=

1

1

1

0

1

l

l

(6)

As it is seen from equations (5) and (6), in this case the main test equations are

independent of the measured quantity x . It should be noted that as the nonlinearity of

67

the transformation function of the initial measuring system increases, it is required to

write its mathematical model with rather high accuracy. When increasing the

accuracy of the measuring by the algorithmic method, the general amount of the tests

contained in MS increases proportional to the order of the approximating polynomial

and at the same time according to the accuracy of composition of tests, the error

components of the measurement results of the tested MS will increase. Furthermore,

by measuring the non-electrical quantities, introduction of excessive tests complicates

its construction or it is not always possible. Therefore, development of an optimal

structure is a very urgent problem. Taking into account the above deficiencies and

restrictions, the tested information-measurement system obtained from the joint use of

additive and multiplicative tests in rather nonlinear TF, will have high accuracy and

stability. For that the non-linear TF of MS is studied along the whole range and

successively separated into optimal intervals (beginning with the zero value of x ).

Unlike the known methods, for attaining a higher accuracy, the greatest curvature,

minimum and maximum points are determined by means of special algorithms. The

curve between extremity points is approximated in two steps and is iterated in other

intervals as well. Such an algorithmic solution of the problem is said to be an

approximation method by the algorithmic-structural way and as a result, a set of

simple curves along the characteristic is obtained. The system of primary equations

describing these curves are jointly solved and we get exact values of the coefficients

of the curve at each interval. This method eliminates the iterative measurement and

reduces the number of the unknown quantities (for each interval) of each

approximating intermediate polynomial to three.

It is known that in the field sx∆ ( ∗= l,...,1s is the number of the approximated

fields) the mathematical dependence between the value of the measured quantity x ,

the tests ( )xAi

( ,,...,1 ei = here e is the number of unknown coefficients in the

approximating polynomial) and their measurement results isy will be as follows:

( )[ ]1

1

−

=

∗∗=

ie

i

esise xAвy (7)

(here ∗∗siв

are the parameters of the TF of MS in −s approximation field). This test

algorithm enables to determine the real result of the measured quantity with high

accuracy at each measurement step using the constant parameters ∗∗

siв . For that on

each quantity sx∆ some contradictory requirements are imposed. On one side, the

quantity sx∆ should be so rather small at each field that the inequality ,.ver.maxsap ∆<∆ ⋅

be satisfied, on the other hand, according to composition conditions of test algorithms,

the measured quantity at the same time should be located at the s approximation

field.

Here .ver.max∆ is the maximum measurement error given by the specifications;

sap⋅∆ is the approximation error of the real TF of the initial MS in the given field. At

this time the current real value of ∗∗

siв may be determined.

68

It should be noted that in the general cases, the quantity sx∆ takes very small

value, 1→sminsmax

xx ; here s

xmax ands

xmin are maximum and minimum values of

the measured quantity x , more specifically, all the intermediate values of the

quantities x and ( )xAi will be located in

maxx∆ .

As 1→sminsmax xx , the isy results of all additional measurements of the tests

( )xAi according to the measurement period s given in approximating field

sx∆ , the

quantity x and the measurement result 0sy will get closer values. Thus, their

difference, in the general case will get small values comparable with the value of the

statistical random error )t(yo

∆ . Therefore, in the process of realization of the

processing algorithm when the initial MS has a rather nonlinear TF final measurement

results may have rather great error.

The contradictory requirements imposed on the value of x∆ make necessary

functioning of IMS in such a working range of initial MS that it could enable to

satisfy them in this or other degree. This condition restricts the use possibilities of all

working range of the initial MS. It should be noted that only the decrease of orders of

the main test equations of IMS equipped with additive tests costs excessive time to

the system. For realizing identification of TF of the initial MS with high accuracy, in

addition to these tests there appears a necessity to additional tests and as a final result,

the ordinary test algorithms providing the full functioning of the system increase from

the provided )n( 1+ number steps to number measurement step.

By the similar mathematical transformations by using the joint use of both tests

we can get a general expression for the measurement algorithm. This algorithm

represents 1−n number additive θ+x test and one multiplicative ),( xk test.

At it is shown, the necessary condition for the existence of the main test equation

(MTE) is the joint realization of simple additive and multiplicative tests in TIMS.

This time, for obtaining minimum order of MTE with respect to it is necessary to

realize 1−n number one kind and one another kind test.

We can write formula (2.3) as follows:

(8)

If we find the values of the parameters na,...,a,a 21 of TF of TIMS from (9) and

write them in (2), we can get a mathematical model of TF of MS. As it is seen from

(8), for 0≠ia by using the total tests of MTE in TIMS we can get an identity with

respect to x . Indeed, assume that for the identification of TF of IMS in the form (1);

in the system the n number additional tests are realized by using the following

combined test:

n2

( )[ ]

( )[ ]jвnв

g

jвnв

gn

j

jxAxA

xxA

yy

=≠≤≤

≠≤≤

= −Π

−Π

=ll

l

,,1

,1

1

0)(

).,.....,1(,)( nвxkxAввв

=+= θ

69

It we write the values of the results of combined test and their measurements

in the expression (8), for the given TIMS we get the following mathematical

expression:

(10)

It is seen from formula (10) that for the MTE of TIMS

realized only on the basis of combined tests has at least one solution with respect to

the measured quantity x .

For nominators of all the summands of the expression (10) are )n( 1− order

polynomials with respect to quantity x and in the denominator the coefficients

are independent of x . Here for all the values 0≠i

a and by using

only the combined tests in the form , the MTE will be )n( 1− order equation

with respect to the quantity x .

So, as in the resulting expression, consisting of a combined test, there are no

number same multipliers with the quantity x , the and don’t become

identities in the domains where the parameters participate.

5 Conclusion

It should be noted that it is possible to increase the accuracy of MR to some extent

by means of the combined test algorithm on the basis of simple additive and

multiplicative tests and application of simple tests in the form and θ−kx

making no difficulties in practical realization of TIOS are more appropriate.

References

1. Bromberg E.M., Kulikovskii K.L. Test methods for increasing accuracy of

measurements. M.: Energia, 1980, 176 p.

2. Kulikovskii K.L., Kuper V.YA. Methods and means of measurements. M.:

Energoatomizdat, 1986, 448 p.

3. Gadjiev Ch.M., Isayev M.M. Methods of processing of measurements in IMM

for determining the mass of oilproducts in reservoirs. Baku, Elm, 2000, 94 p.

4. Isayev M.M. Metrological supply of control and measurement of the quantity of

oil products in commercial units. Izvestia Vyshshikh Tekhnicheskikh uchebnikh

zavedeniy Azerbaydzhana. Baku, 2008, No 1 (53), pp. 65-68.

5. Melikov Ch.M., Isayev M.M. Pressure measuring equipment. Patent No 99

(001607, class G 01 L 19/00, Industrial property. Official bulletin. 200l, No 1.

jy

( )[ ]

( ) ( )[ ].

1

.,,1

,1

1

0

jllвnв

lвlв

jвnв

вВn

j

jkkx

kx

yy

=≠≤≤

≠≤≤

= −+−Π

+−Π

=θθ

θ

1,0,0 ≠≠≠ввв

kkθ

( ),,....,1 nj F θθµ =

)(xAв

1−nв

θ k

θ+kx

70

Constructive & Functional Representation of Engineering Products in a CAD-system

at the Stage of Technical Design

D. Tsygankov, A. Pokhilko, I. Gorbachev

Ulyanovsk state technical university, Ulyanovsk, Russia e-mail: d.tsyg@mail.ru

Abstract. The paper presents an approach to automation of design activity, based on the bijective conformity between the designed product structure and the process of building its electronic 3D model. The main difference of the proposed approach is the fixation of the design intent in the tasks of forming and modifying the design solution in the form of an electronic 3D-model with saving of its semantic integrity.

Keywords: engineering product construct, 3D-model, semantic fullness, CAD-system, technical design process, constructive structure, modular principle, constructive element, semantic similarity.

1 Introduction

The continuous development of information technology in manufacturing sector strengthens the position of electronic 3D-models in the product life cycle. This applies, first of all, to the stage of development work, at which the 3D-model fully represents a design solution [1].

It is obvious that at the stage of technical design (construction), the most important information is contained in the product design displayed by CAD-systems as follows:

(Prod),Modd)Constr(Pro:CAD 3D→ (1) where Constr and Mod3D – construction and 3D-model of designed product Prod.

However, in the process of forming the product design in the CAD-system, the designer operates only with the abstract functionality of the CAD-system used, which leads to the loss of the original design intent. As a result, the possibilities of reuse and modification of the generated 3D-model are limited. Thus, fixing the design intent in the 3D model is an urgent task.

2 Designed Product 3D-model

In modern CAD-systems, product 3D-model is only a “consequence” of the implementation of basic operations (BO) – the simplest design operations provided

by the functionality of the used CAD-system [2]. Basic operations are recorded in the so-called the "creation tree" of the 3D-model, which is a linear sequence of interconnected BO, then:

(Prod),ModBО:CAD 3Dn

1ii →

=

(2)

where the union symbol means the sequence of execution of the BO that forms the design route – a set of BO provides for the construction of a 3D model of the product.

In the framework of the Constructive Solid Geometry technology, implemented in almost all modern CAD systems, the 3D model can also be represented as a system:

(Prod),ModCGE:CAD 3Dm

1kk →∑

=

(3)

where CGE is a structural element of geometry, defined as “an object with a predetermined behavior and data structure specified during the design procedure”. The equivalent of CGE in English literature is the term "features".

3 3D-model informativeness

The information content of the 3D-model consists in displaying the product information required for the current stage of the life cycle. The display of its design is the main functionality of the CAD-system; design solution in the form of a 3D-model has a complete design. Moreover, all the design data displayed by the 3D-model is contained in the basic operations hierarchically ordered in the creation tree:

,BOd)Constr(Pro:)Tree(Modn

1ii

3D

=

→ (4)

where )Tree(Mod 3D – 3D-model creation tree. The construction tree fully describes 3D-geometry and is the main source of information about it.

The most informative 3D-model at the stage of technical design is achieved by displaying the design structure of the product. Such a 3D-model is already fully a component of the digital product layout.

The national standard of the Russian Federation GOST R 53394-2017 defines the engineering product design structure as “a structure (system) consisting of structural and functional elements, as well as the relationships that determine their mutual occurrence”. Such a systematic definition of the structure of the product corresponds to the modular principle

4 Engineering Products Design Construction

Constructive elements suitable for the above definition include: Operating elements performing the functions regulated product; Basic elements that coordinate one product relative to others; Connecting elements, materially connecting products during the assembly process

with each other;

72

Technological elements that implement the technological process of manufacturingthe product and its subsequent assembly.

The same CE can play the role of both working and basic and connecting elements; the most favorable option is the combination of basic elements in the design of work elements while minimizing the number of connecting elements.

Each constructive element has semantic fullness, consisting in constructive and functional purpose, partially present in its decimal number and name [3]. Therefore, CE has a meaning in a given subject area. The subject area is determined by the final product of the “Assembly Unit” level. Based on the physical meaning, the name of the CE is set and the attributes that determine its 3D-information image are highlighted; their values are characteristic of a specific CE instance.

Taking into account the structural and functional decomposition of the product design on CE and the composition of the 3D-model from the BO sequence (see expression (2)), the display of the tree for constructing the view mjni CEBO → in a CAD-system can be implemented by two different methods.

5 Structural Conformity Principle

The principle of structural correspondence when displaying constructiv structure in the creation tree of 3D-model building is based on the following three criteria,:

1. Fixation and display of design intent within the 3D model;2. Operation with semantic units relevant in a given subject area in the process of

forming a 3D model; 3. Ease and convenience of the process of building and reusing a 3D model.The structural conformity principle proposed by the author [4] is, in essence, a

bijective display of the design structure of the designed product (as a set of CE) in the tree of construction of its 3D-model, which is shown in Figure 1 and formula (5):

,n1,g ,CESMOSMO)Tree(Mod ggi3D =→= (5)

where n – is the number of CE in the constructive of the product, SMO – is a semantic macro operation of constructing a 3D-object corresponding to one CE, defined as:

m

1jji ,BOSMО

=

= (6)

those SMO is an ordered BO sequence that forms the resulting 3D-object of the structural element.

The main idea of the principle of structural conformity an informational and semantic composition of the basic operations of the CAD-system according to formula (6) up to the level of semantic macrooperation with strict correspondence of the form SMO → CE. So, the design structure of the Constr(Prod) product and the tree of construction of the 3D-model Tree(Mod3D) are equally powerful as sets, i.e.:

.)Tree(Modd)Constr(Pro 3D= (7)

73

Figure 1. Display the design structure of the product in the its 3D model creation tree

Thus, advantages of the approaches are realized in a one-to-one correspondence between SMO and CE, which contributes to the fixation of the design intent in the process of constructing a 3D model.

6 Example of Constructive & Functional Representation

Figure 2 shows the proposed display of the product structure of the “Part” level – the case from the composition of the microstrip UHF module.

Figure 2. Structural-semantic representation of the “Case” part As can be seen in Figure 1, the tree construction of the 3D-model is no longer a

sequence of abstract BO, but carries a fixed design meaning, reflecting the structure of the designed product.

Representation of a 3D-model in the form of a SMO sequence provides structural-geometric variability of the design solution. This is achieved by referring to the SMO as an independent module, associatively associated with the others, which allows to maintain the semantic correctness of the design.

74

7 Semantic Variation

The structural and geometric variability of 3D-models of parts included in the assembly unit allows to specify a class of semantically similar design solutions. The rules of behavior set by the SMO to tie components not to abstract geometric, but to structural elements of each other, both in the task of forming and modifying an assembly 3D-model.

8 Conclusion

The author’s proposed methodology for representing a 3D-model in CAD-system is aimed at increasing the information content of the latter while reducing the complexity of building it and the time it takes.

Moreover, this approach does not operate with structural elements of geometry that are tied to a specific CAD-system, but restores the design structure of the designed product based on semantic macro operations. It is SMO that are the main design action that allows you to get away from abstract geometric elements in the description of the 3D-model to the real structural elements of the designed product.

The proposed approach has shown its effectiveness only when working with 3D-models that are distinguished by high geometric complexity. Such models correspond to products with a complex functional structure; These include, as an example, components of microwave UHF modules.

The next step is to develop a library of semantic macro operations for a specific class of designed products, which provides the functions of creating, storing, calling, editing and deleting SMO. If, from the point of view of the user-designer, SMO are integral (indivisible) units, then in this library their internal structure and rules of behavior will be set.

This work was supported by the RFBR, research project 18-47-730028.

References: 1. Tsygankov D., Pokhilko A., Gorbachev I., The Designed Product Construction

Information Semantic Representation in a CAD-System, Transdisciplinary Engineering Methods for Social Innovation of Industry 4.0 – Proceedings of the 25th ISPE International Conference on Transdisciplinary Engineering, IOS Press, Amsterdam, 2018, pp. 1092-1101.

2. Tsygankov D., Pokhilko A., Gorbachev I. CAD-system Basic Operations SemanticGeneralization to the Designed Product Construction Conformity, Transdisciplinary Engineering: A Paradigm Shift: Proceedings of the 24th ISPE International Conference on Transdisciplinary Engineering, IOS Press, Amsterdam, 2017, pp. 603-610.

3. Tsygankov D., Pokhilko A. The Product Design Information Imaging at theConstruction Stage in 3D-model Creation Tree, Procedia Manufacturing, Vol. 11, 2017, pp. 2069-2076.

4. Tsygankov D., Pokhilko A. Designed Product 3D-model Semantic Representation ina CAD. Interactive Systems : Problems of Human - Computer Inter-action, Collection of scientific papers, Ulyanovsk : USTU, 2017, pp. 255-259.

75

Identifying Scientific Constructs in the Results of Question-and-Answer Reasoning

to Support Project Theorizing

A. Kulikova, E. Trifonova

Ulyanovsk state technical university, Ulyanovsk, Russia e-mail: a.push1206@gmail.com, catherine21tea@gmail.com

Abstract. The article considers the problem of theorizing in software engineering and describes modules that provide developing of a project theory by structuring the units extracted from designer’s reasoning at the conceptual stage of designing the project.

1 Introduction

Currently, a range of problems occurs in developing software-intensive systems (SISs). One of the major problems is a low degree of success: according to the reports of Standish Group [1], only 16% of projects were successfully completed in the mid-1990s, now this indicator has reached 40%; nevertheless, this cannot be estimated as satisfactory.

The most significant factor influencing the success rate is a phenomenon of understanding. Inappropriate understanding of a task and the process of solving the task by a designer lead to expensive semantic mistakes in reasoning and documentation, and, accordingly, in project solutions and their direct implementation.

Therefore, one should register acts of understanding and integrate them in systematic formations. In practice, certain theories are used for this purpose. The authors of papers [2] and [3] note that at present, there is a problem of theorizing in the field of software engineering and its applications, which is characterized by the lack of fundamental theory on software engineering and insufficient practice of applying approaches and practices from the scientific and theoretical activities. For this reason, the degree of success in developing SISs decreases, especially at the design stage. Thus, theorizing should aim at creating a project theory mainly used in the design process.

Considering the abovementioned, it is suggested to structure units of project reasoning for their further use in building the project theory. For this purpose, a special module is being developed in the designing toolkit OwnWIQA [4]. The module helps to assign to question-answer units (QA-units) that contain information about the designer’s reasoning a certain type of theoretical construct and to use special icons for the visual representation of the information.

2 Related works

Due to the global, rapid development of software engineering, the interest of researchers to the role of theory in this sphere has grown. The problem is that the general theory of software engineering is still missing. In paper [5] the authors claim that “to become a real engineering science, software engineering needs to develop stronger theoretical foundations”.

In recent years, some researchers have started to realize the expediency of building applied Grounded Theories [6] for certain organizational and behavioral phenomena in software engineering applications. One of the reasons for this intention is related to positive effects in some areas of designers’ activity, but other researchers believe that the way to the General Theory can be found by means of applied theories (the way from applied Grounded Theories to the General Theory of software engineering).

There are several kinds of Grounded Theories and approaches to building a theory based on the method of developing grounded theories. One of such approaches to the creation of the theory of software engineering and project activity is a substantial evolutionary approach presented in [2].

The approach is based on a specialized system, the structure and semantics of which are determined by the reflection of the operational design space in the semantic memory of the “question-answer” type (QA-type). This reflection is based on question-answer reasoning and its protocoling, which is an important source of facts disclosing the interaction of designers with experience and its models during the implementation of a certain project.

The substantial evolutionary approach to creating a theory for the project also uses the design thinking (DT) paradigm proposed by K. Dorst [7]. According to this paradigm, the design process should be split into 5 phases: empathize; ideate; define; prototype; test.

Accordingly, the process of creating a substantial evolutionary theory (SE-theory) also has several phases, which go in parallel with some other project design activities: before-theoretical phase; descriptive phase; classification; identificational and measuring phase; formal phase; model phase.

3 Scientific Contsructs Identification

One of the main features of the substantial evolutionary approach is that a theory of any project begins developing from the first steps of designers’ conceptual activity. The facts extracted from QA-memory or, in other words, from the question-answer models formed in the process of interaction with experience, then are processed to the form the project language. The project language systematizes concepts related to the project and manages the conceptual activity of a designer in order to avoid mistakes in reasoning and project documents. In addition, the processing of facts provides theoretical support for the project, which, as a result, forms a language of the project theory.

77

The term “theory” defines a certain system of text units. The system, in turn, implies the presence of some elements in it and relations between them. The analysis of the structure of some major theories in software engineering led to the conclusion that any theory is expressed by a set of theoretical constructs of different types (such as principle, hypothesis, statement, etc.) that can be used as project units.

After extracting facts at the before-theoretical phase of developing theory, a designer can see that the facts have different semantic and theoretical content. Therefore, it is possible to assign to a QA-unit some type of theoretical construct. Thus, the combination of all units collected after processing of facts can be a basis for the project theory.

Each type of scientific constructs should have its own visual mark, and, as a result, a question-answer protocol with QA-units of various theoretical types becomes marked. This feature facilitates navigation through the project content.

For further automation of the work with the types of theoretical constructs, a special dictionary was created in the toolkit OwnWIQA. The dictionary contains 75 entries. Every entry includes the following fields:

• ID;• name;• definition;• icon.

Let us consider some of them as an example (see Table 1).

Table 1. Scientific constructs (example)

ID Name Definition Icon 101 Abduction Cognitive procedure of hypothesizing.

102 Analogy A comparison of two otherwise unlike things based on resemblance of particular aspect: resemblance in some particulars between things otherwise unlike,

103 Analysis

1) A detailed examination of anything complex in orderto understand its nature or to determine its essential features: a thorough study; 2) separation of whole into its components.

104 Axiom

1) A statement accepted as true as the basis for argumentor inference; 2) an established rule or principle or self-evident truth; 3) a maxim widely accepted on its intrinsic ment.

105 Class A set of objects related by a common structure and behavior for a set of similar objects representation of which are called class elements.

Further, it is proposed to identify such texts of project reasoning that can be attributed to the theory of the project and subsequently become the basis for its theorization. For this reason, a special module is developed in the toolkit OwnWIQA. The module allows us to extract the question-answer units (QA-units) that store information about the designers’ reasoning in the course of work on the project (P),

78

which are marked with a special attribute indicating its relation to the theory, and to save them in a separate project (PT). Figure 1 schematically depicts this process.

Figure 1. Identifying Scientific Constructs Process

Let us consider how a designer can mark a QA-unit, which in his/her opinion is related to the theory of a project, in a more detailed way. To simplify this activity, we created a separate attribute (Theory Consctruct Type) which can be added to any QA-unit and have one of the 75 values depending on its type (see Figure 2).

Figure 2. Theory Construct Attribute

Right-click on a QA-item invokes a context menu where a user can choose the option to add or edit its attributes. Figure 3 shows the Attribute Explorer window for

79

the A13.4.1 item which has the Theory-Construct-Type (TConType) attribute with the Hypothesis value. That means that the current QA-unit is considered to be one of the hypotheses for the project theory.

Figure 3. Theory Construct Type Attribute Example

After all the necessary QA-units are marked with the corresponding attributes, the following procedure should be launched:

1) IF there is no PTX project for the PX project THEN create a PTX project ELSE;2) i = 1;3) IF QAi item has a ATY attribute THEN add QAi to PTX with the IY icon;4) IF i == QANum(PTX) THEN GO TO point 7 ELSE;5) i = i + 1;6) GO TO point 3;7) FINISH.As a result, all the QA-units marked with the TConType attribute will be added to

the separate project related to the project theory.

7 Conclusion

The proposed modules allow us to structure project units extracting theoretical constructs of different types in order to form a project theory from them.

Further investigations in this sphere can include creating methods to identify the construct of a certain type; finding out which features each construct type may have and how this information can help to automate the reasoning process further.

The creation of project theory is expected to resolve the problem of theorization in software engineering that will lead to a higher degree of success in developing projects.

80

References: 1. Chaos reports 1994-2017. Available at: http://www.standishgroup.com.2. Sosnin P. Substantially Evolutionary Theorizing in Designing Software-Intensive

Systems / Information 2018, 9, 91. Availabe at: https://www.mdpi.com/2078-2489/9/4/91. 3. Sosnin P. A way for creating and using a theory of a project in designing of a software

intensive system. In Proc. of the 17th International Conference on Computational Science and Its Applications (ICCSA). 2017. pp. 3–6.

4. Sosnin, P. A Scientifically Experimental Approach to the Simulation of DesignerActivity in the Conceptual Designing of Software Intensive Systems. IEEE Access 2013, 1, 488–504.

5. Stol, Klaas-Jan & Fitzgerald, Brian. (2013). Uncovering Theories in SoftwareEngineering. 2013 2nd SEMAT Workshop on a General Theory of Software Engineering, GTSE 2013 - Proceedings. 10.1109/GTSE.2013.6613863. Available at: https://www.researchgate.net/publication/258518078_Uncovering_Theories_in_Software_Engineering

6. Barney G. Glaser, Anselm L. Strauss. The Discovery of Grounded Theory: Strategiesfor Qualitative Research. Transaction Publishers, 2009.

7. Dorst, K. The Nature of Design Thinking. In Proceedings of the Design ThinkingResearch Symposium, Sydney, Australia, 19–20 October 2010; pp. 131–139. Available at: http://epress.lib.uts.edu.au/research/handle/10453/16590/.

81

Formation of requirements to the modeling

architecture of the automated system.

A. Podobriy

Federal Research-and-Production Center Joint-Stock Company ‘Research-and-Production

Association ‘Mars’ ( FRPC JSC ‘RPA ‘Mars’), Ulyanovsk, Russia

e-mail: alexandr073@mail.ru

Abstract. The article deals with the approach to the formation of requirements

for modeling the architecture of an automated system (AS). The question-and-

answer analysis from the position of success of development of as and the

arising problems is carried out. The author presents the structure of AS

architecture description from the perspective of stakeholders ' interests.

1 Introduction

Every year the variety and complexity of systems, which have received in the

international scientific and technical practice the name of systems, intensively using

software (Software Intensive Systems, SIS), increases [1]. In systems of this kind, the

functional potential is determined by the software (S) or depends on the SOFTWARE

to a significant extent. Generally recognized legislator in the field of research and

development SIS Institute of software engineering Carnegie Mellon University

(Software Engineering Institute, SEI) refers to the class SIS system in which software

modules represent a significant segment on the following items: the functionality of

the system, its cost, risks in the development process, development time. In such

systems, software components interact with each other and components and

subsystems of a different nature, sensors, devices and people involved in the use of

SIS [2].

According to The Standish Group about 65% of IT projects around the world are

considered a failure. It would seem, why in the era of Agile development the failure

rate is so high? Even when projects fit into the schedule and budget, they can fail

regardless of the breadth of the provided functionality, if they do not give the results

that end users expect. And these are the expectations and interests of users for which

the system is developed. According to the report of the Project Management Institute

of 2017, prepared on the results of a survey of 3 thousand. project management

specialists, 28% of the strategic initiatives they led, were absolutely failed.

Approximately 37% of respondents cited the lack of clearly defined and achievable

milestones and targets used as an assessment of project completion as the reasons for

the failure. Further, a number of reasons were lack of information exchange (19%),

lack of feedback from senior management (18%), resistance of employees (14%) and

insufficient funding (9%) [3].

The complexity of the systems being developed today has reached an

unprecedented level. This has opened up new opportunities and, at the same time, has

complicated the problems for the organizations that create and use such systems. For

the control of complex systems, it is necessary to apply the concepts, principles and

procedures of the process of architecturial [4].

Description of the system architecture contributes to the understanding of all

stakeholders about the structure, basic properties and methods used in achieving the

stated interests.

2 Existing approach

Not all software systems are complex. There are many programs that are

conceived, developed, maintained, and used by the same person. Usually it is a

beginner programmer or a professional working in isolation. It is impossible to say

that all such systems are poorly made or, moreover, to doubt the qualification of their

creators. But such systems tend to have a very limited scope and short lifetime. The

development of such programs is rather tedious than difficult, so we are not interested

in studying this process. Formally, it is not possible to distinguish an ordinary

program from a complex one. Complex programs often differ in the variety of

services provided and the amount of information processed. It is possible to

distinguish only some qualitative characteristics peculiar to the difficult program.

Today there is a group of ISO SC7 standards in the field of software and

engineering systems that meet market and professional requirements. These standards

describe processes, tools and supporting technologies for software and system design,

and are primarily focused on process models and best practices [6].

In the field of system architecture description the standard ISO/IEC/IEEE 42010

Systems and software engineering — Architecture description, domestic localization

GOST R 57100-2016 "System and software engineering is included. Description of

architecture". The standard, designed to manage the architectural description of

complex systems, allows you to describe the structure of the automated system taking

into account the interests of stakeholders.

There are other standards, such as a series of GOST 34 and 19, for the

development of AC and software documentation and others. The presented state

standards fix the requirements for the developed functionality of the system, but do

not participate in its modeling architecture.

3 Question-answer models

As mentioned earlier, the description of the structure of the AU is necessary first

of all to increase the level of success of the AU development. By success we mean the

development of the AU taking into account the stated requirements in the allotted

time and budget. Let's try to form a question-answer analysis to identify the necessary

requirements Of the s system from the point of time (Figure 1).

Q1. Why increase the development time of the AS?

83

A1.1. Due to incorrect work planning.

A1.2. Due to lack of necessary resources.

A1.3. The lack of interested parties.

Q1.1. Why are the works poorly planned?

A1.1.1. Due to incorrect requirements or lack of description of requirements (A).

A1.1.2. Due to lack of development experience.

A1.1.3. Insufficient information for planning.

Q1.1.1. What information should contain the requirements for the development of

the AS?

A1.1.1.1. Description of the objectives of the developed as.

A1.1.1.2. Description of the functions of the developed as.

A1.1.1.3. Description of qualitative characteristics and assumptions.

A1.1.1.4. Description of the technologies used.

Q1.1.3. How to understand that the requirements for the developed as are

achievable?

A1.1.3.1. There must be measurable criteria for each requirement.

A1.1.3.2. Each requirement must be materialized in the form of links to other AS

objects.

A1.1.3.3. Each requirement must be visualized in one of the views.

Q1.2. How to analyze the required resources?

A1.2.1. You need a description of the resource list that is bound to the AS function

blocks.

A1.2.2. Analysis of the probability of resource exhaustion.

Q1.3. Why are there no stakeholders?

A1.3.1. Due to the lack of evaluation of stakeholders and their impact on

activities.

A1.3.2. Lack of motivational factors.

Q1.3.1. What is needed for stakeholder assessment?

A1.3.1.1.Description of all participants involved in or associated with the

development of the AS (B).

A1.3.1.2.Determination of the level of interest of each participant.

A1.3.1.3.Communication of participants with stages of work (functions and

characteristics) and elements of AS (C).

Figure 1. QA time-based analysis.

To control the timing of execution, the architecture S must include A, B, C, where

A - definition of architecture description and overview;

B - identification of system stakeholders and their interests;

C - define each point of view on the architecture used in the architecture

description.

From the standpoint of the cost of development requirements can be formed as

follows Figure 2.

Q2. Why does the cost of development increase?

A2.1. Due to the ambiguous interpretation of the requirements between the

customer and the contractor.

84

A2.2. Due to the change of requirements from the customer.

A2.3. Due to the obsolescence of the required technologies.

Q2.1. What is necessary for the requirements to be clear to both the customer and

the contractor?

A2.1.1. Formalization of requirements at all levels.

A2.1.2. Visualization of requirements at all levels (D).

Q2.1.1. At what levels should the requirements be formalized?

A2.1.1.1. Functional structure of the system.

A2.1.1.2. Information structure.

A2.1.1.3. Process level.

A2.1.1.4. Procedural level.

Q2.1.1.1. What areas does the functional structure describe?

A2.1.1.1.1 Functionality, external interface, internal structure, design

characteristics.

Q2.1.1.1.1 What information does the functional level include?

A2.1.1.1.1.1 Functional elements, connections, interfaces, constraints,

relationships.

Q2.1.1.2. What problems arise in the description of the functional structure?

A2.1.1.2.1 Poorly defined requirements and interfaces.

A2.1.1.2.2 Description of functional elements without reference to infrastructure

(E).

A2.1.1.2.3 Overloaded or informal presentations.

Q2.1.1.3. What areas does the information structure describe?

A2.1.1.3.1, the data storage Structure, database relationships, actions with the

data.

Q2.1.1.3.1 What information includes information level?

A2.1.1.3.1.1 Entities, constraints, relationships, use of objects (F).

Figure 2. QA analysis from the standpoint of cost.

To control and analyze the cost of development, it is necessary to include elements

D, E, F in the structure S, where

D - architecture views;

E - architectural models for each architecture point of view used;

F - applicable communication rules in the architecture description, communication

and registration of known inconsistencies in the required content of the architecture

descriptions.

Thus, the task of the developed system is to achieve the stated interests of An in

the form of a specific implementation of Wn.

The system architecture (S) includes the following areas:

S:=A,B,C,D,E,F,G.

Figure 3 shows the interconnection diagram of the s system architecture.

85

Figure 3. System architecture.

Architecture description A:=A1,A2,A3,..,An, where An-interests, should

identify (identify) the system in question and include additional information as

defined by the project or organization. The detailed content of identification and

additional information objects should be specified by the organization or project.

In the role of interests, as already mentioned, are:

• the objectives of the system;

• the suitability of the architecture for achieving system;

• feasibility of system design and deployment;

• potential risks and impacts of the system on its stakeholders throughout its life

cycle;

• maintainability and development of the system.

The description of the architecture should identify stakeholders

B:=B1,B2,B3,...,Bnsystems that have interests that are important to the architecture

of the system under consideration. The architecture description should take into

account the following stakeholders:

• system user;

• system operator;

• acquiring parties of the system;

• system owner;

• system suppliers;

• system designer;

• system builders;

86

• accompanying parties of the system.

The architecture description should include every point of view used

C:=C1,C2,C3,...,Cn on the architecture. Each considered point of view on

architecture should be defined. Each interest should be structured with at least one

point of view.

The architecture description should include only one architectural representation

D:=D1,D2,D3,...,Dn for each architecture point of view used. Each architectural

representation must adhere to the conventions of its main point of view on

architecture.

The architectural representation must be composed of one or more architectural

models E:=E1,E2,E3,...,En. Each architecture model shall include version

identification as specified by the organization or project. Each architectural model

must define its main type of model and adhere to the agreements of this type. An

architectural model can be part of more than one architectural representation.

The architecture description should contain the registration of any known

inconsistencies through the architectural models and representations

F:=F1,F2,F3,...,Fn. Relationships and rules can be used to record changes and

analyze the consistency between models, views, and other elements within an

architecture description.

Each link in the architecture description needs to be defined and to describe the

interaction between the elements of the description of architecture. Elements of

architecture description can be any constructions: stakeholders, interests of the

system, points of view of architecture, architectural representations, etc.

4 Conclusion

Ensuring the integrity of large information systems requires maintaining the

system architecture up to date at all stages of development. The most correct solution

is to divide the design of automated systems into two stages: the General design stage

and the design stage of individual subsystems. At the stage of General design the

architecture of the system necessary for the customer is formed as a logical model of

the automated system that combines the model of business processes, information

model and organizational structure model. At the stage of detailed design of

individual subsystems, the system architecture is refined for the developer, which

ensures that the system models are kept up to date.

Description of the architecture of the automated system is one of the important

tasks, the solution of which will allow to control the development and changes of the

system. Analysis of existing problems and risks in the development of complex

systems enables the designer and the developer to design the system correctly.

Monitor changes that occur in the course of work and analyze their impact on all

existing modules. To form precedent– based solutions in the design of automated

systems. The distribution of the design into several iterations from a larger to a more

detailed structure allows all stakeholders to be involved in order to take into account

the interests of all users of the system.

References:

87

1. ANSI/IEEE 1471-2000, Recommended Practice for Architecture Description of

Software-Intensive Systems

2. Software Engineering Institute // http://www.sei.cmu.edu

3. Electronic journal "Director of information service", 2017, 07, Why it projects still

fail.

URL -https://www.osp.ru/cio/2017/07/13052958/

4. ISO/IEC/IEEE 42010 Systems and software engineering — Architecture description

5. Official site «International Organization for Standardization»

URL - https://www.iso.org/committee/45086.html

6. Bonneau V. at all., Software intensive systems in the future//

ftp://ftp.cordis.europa.eu/pub/ist/docs/embedded/puissochet-060207_en.pdf

88

Adjustment of Coordinate System Using Skeleton Models of Parts and Assembling Units of Aircraft Pipelines

P. Pavlov

Ulyanovsk state university, Ulyanovsk, Russiae-mail: pavel.y.pavlov@mail.ru

Abstract. The article discusses skeleton models of parts and piping systems of aircraft. An important feature of these models is their application for adjusting coordinate systems at various stages of the life cycle. Such coordination can be realized with the introduction of feedback in the form of a laser tracker and correction factors for the welding robot.

1 Introduction

Today high-tech production characterized by active application of computers and programs, such as PLM and CAD systems. Also, a large number of people are involved in product life circle. In other words, such an activity can be characterized by collective activity with wide computers using. In this situation there is a problem of accumulating and reuse of professional experience.

In the practice of producing piping for aircraft systems, there is a problem of geometric modeling, coordination and control of the manufacture of piping assemblies. There is also the problem of their conceptual presentation and accumulation of experience in the form of useful precedent models. The product description in the form of such precedent models defines the conceptual space of the ontology, the coordinates of which are attributes, properties, and semantic dependencies.

For increasing the efficiency of accumulating professional experience and using it at all the main stages of the design and technological preparation of production, it is necessary to develop tools to link the conceptual space of ontology with the coordinate space of the aircraft. Such a coordinate space can be represented both in 3D models that are stored in a special database and in the form of physical embodiment in the form of manufactured parts that need to be assembled and controlled. To control geometric deviations, various measuring equipment can be used, which is connected with 3D models: control and measuring machines, laser trackers, 3D scanners, etc. The use of such equipment in conjunction with a life cycle management system allows us to talk about the presence of a cyber-physical system.

Based on the above, there was a need to develop such a toolkit that would allow us to connect the geometric space of parts and assembly units with the conceptual space of ontology for more efficient accumulation of professional experience.

2 Preliminary Bases

For the design of parts and assembly units of the aircraft is actively used artifact, which is called as master geometry (MG). It sets the aerodynamic shape and defines the geometric space in which all the components of the aircraft are located. This artifact has computerized representation and can be used by the design team for different various aims, for example, when they create doubly curved aircraft covering [2].

Sample of the MG represented on figure 1. The MG-space it allows for engineers to simulate the pre-installation of aircraft subsystems and various pieces of aircraft equipment. During such work, step by step, the MG-space are filled by components of aircraft sub-systems.

The MG also helps technologists in developing of production processes for manufacturing the aircraft components, and it also enables to assess how the maintenance operations can be performed on exploitation.

Figure 1. Structure of the MG

Since a modern aircraft is a very complex technical product, this article will be limited to the scope of metal parts and piping systems and the welding process using a robot. Moreover, it is assumed that for the pipeline to be manufactured, the tasks of the arrangement of its parts in the assembly units and their placement in the «space» (figure 2) of the aircraft have already been solved. So, for each particular part, its positioning is fixed to the geometry of the aircraft.

In this case, any pipeline part and any their assembly can be placed in the aircraft space, and any point of the pipeline construction can be specified by its coordinates in the MG- system. Furthermore, a designer or engineer or technologists can present any part of their assembly in the own system of coordinates that binds with the MG by linear transformations. The welding robot also works in own coordinate system and there is a problem of adjusting various coordinate system.

Figure 2. Model of pipeline in the aircraft space

90

3 Related Works

The aircraft development process is a multidisciplinary activity based on the intensive use of various geometric models of parts, assemblies and their complexes at different stages of the life cycle [1]. That is why such an artifact as MG is used in the design and production processes as a single source of information. MG can also be shared for parallel solving a wide range of tasks in real time.

First of all, MG is a source of geometric constraints [3], which relate to certain areas of the aircraft in space. To facilitate interaction with such areas, MG is often divided into parts [4, 9], which are focused on the use of MG content in accordance with the design and technological division into compartments and assemblies, as well as for certain aircraft subsystems or life cycle stages. In [6], its authors propose using a special “buffer” geometry as a model for the transition and monitoring between the external geometry of the aircraft and the MG.

Among the typical applications of MG or their components, geometry control during aircraft development is noted [7]. From a conceptual point of view, the geometry of the aircraft and its use in conceptual design are carefully considered in the article [5].

Also skeleton models was reviewed in [8] and adjustment of coordinate system was reviewed in article [10].

4 Skeleton Models of Parts and Assemblies

Based on the foregoing, let us consider in more detail the skeleton model of the parts and assemblies of the piping systems of aircraft.

Skeleton model in the form of a broken set of segments with orientation along the axes of the pipeline parts and with the coordinates of the ends in the absolute coordinate system of the aircraft.

To unambiguously determine the coordination of the pipe in space, it is necessary to distinguish three points located on the inner diameter of the pipe wire. The choice of the inner diameter is due to the fact that liquid or gas is transferred through the piping and, therefore, for normal flow there should be no protrusions and steps, while the shape of the outer surface can be arbitrary, since it does not affect the flow liquid or gas.

For parts of the “straight pipe” type, the skeleton model of each of them will have the form of a segment limited by the plane of the joints (Figure 3)). Such a geometric representation will be useful to use for the formation of precedent models that are tied to the stage of designing parts and piping assemblies of aircraft systems, and thereby connect the design stage with the remaining stages of the life cycle.

The choice of three joint points allows us to uniquely specify the position of the end face of the connection of two parts, united along the pipeline route in space in the form of a plane equation, which in general can be represented as

AX + BX + CY + D = 0,

91

where A, B, C – constant coefficients, and X, Y, Z – coordinates, and which has a solution in the form of a system of linear algebraic equations.

Let explain by the example of the connection of two parts such as “straight pipe” and “flange” the sufficiency of the selected version of the representations of parts to verify the correctness of their joint. Such an explanation is valid for any parts and assemblies in places of their flat joint, that is, such a joint in which the circuits restricting the flows in the pipelines for the mating parts coincide.

Figure 3. Skeleton models for a liner part Pi

Sufficiency has two dimensions: 1. The joint plane for the first part should overlap or coincide with the junction

plane of the second part at the junction in the specified coordinate system.2. The diameters for which the angular characteristics of the coordinates are the

same must match.The skeleton model of the straight pipe assembly is shown in figure 4.

Figure 4. Skeleton models for an assembly unit (Pi-Pj)For ease of use, the skeletal model can be presented in the form of a list view,

which contains sufficient information to describe the end of the piping part in space and for a part like “straight pipe” will have the form:

det_numberi; connection_typei; Di; Pi1, Pi2, Pi3, $ det_numberj; connection_typej; Dj; Pj1, Pj2, Pj3.

92

Such a representation of the end face of the part in the form of a mathematical equation allows us to describe the location of the connected end face of the first part of the relative end face of the second in space after measuring the specified points on the manufactured part and solving the equations for finding the angle between planes and distances from point to plane. These equations have long been known, have implementation in most modern CAD systems, so they are not of interest and will not be given. And the application is associated with checks of defects such as geometric defects in pipeline welding.

For the given node, the verification algorithm consists of the following sequence of actions:

1. The choice of the coordinate system, for example, the coordinate system of thepart Pi.

2. Representation of points (Pi4, Pi5, Pi6) and (Pj1, Pj2, Pj3) in the selectedcoordinate system.

3. Calculation of parameters A1, B1, C1 and D1 for the plane passing through thepoints Pi4, Pi5, Pi6.

4. Calculation of the parameters A2, B2, C2 and D2 for the plane passing through thepoints Pj1, Pj2, Pj3.

5. If the parameters of the expressions for both parts are equal and the centers ofthe circles coincide, then the arrangement of the parts relative to each other is correct or is within the tolerance.

It should be noted, the last point in enumerated actions defines the rule of checking for parts, the sections of which are circles, and the diameters of sections are the same. If sections are ellipses, then the rule of checking will be different. Rules of checking the correctness of positioning are depended from contours of parts to be welded.

5 Adjustment of Coordinate System

Aircraft is a complex product with a large range of parts, assemblies and subassemblies. In a modern medium-haul aircraft, the number of original parts can reach 300,000 or even more. Electronic models of all parts are stored in a special database, which is controlled by the product data management system or the product life cycle management system, and each specific part is allocated a certain position in the space of the MG.

This position of the part can be described by the coordinates X, Y, Z in the framework of the aircraft MG and, of course, its absolute coordinate system. For each i-th part of the pipeline, we define its own coordinate system associated with the MG by the following linear transformation:

0

0

0

,,,

i i

i i

i i

X X AY Y BZ Z C

where X0, Y0, Z0 – the coordinates of the i-th part in the absolute coordinate system of the aircraft; Xi, Yi, Yi – coordinates of the beginning of the i-th part; Ai, Bi, Ci – linear

93

dimensions of the origin of the absolute coordinate system and the origin of the i-th part.

Such a relationship of the coordinate system of the pipeline part with the absolute coordinate system of the aircraft is shown in Figure 5, where Toi(Ai, Bi, Ci) is a 3 × 1 matrix, which is the vector of the position of the beginning of the pipeline part relative to the absolute coordinate system.

Figure 5. The relationship of the coordinate system of the i-th part with the absolute coordinate system of the aircraft

Since the part in figure 5 is connected with the assembly details of the assembly at its ends, in this coordinate system you can represent the coordinates of the connection, for example, with the coordinate system of the j-th part (figure 6):

,,,

j i j

j i j

j i j

X X AY Y BZ Z C

where Xj, Yj, Zj – coordinates of the beginning of the j-th part relative to the i-th part; Aj, Bj, Cj – linear displacements of the origin of the j-th part.

In matrix form, the above expression can be written in the form Tij(Aj, Bj, Cj).

94

Figure 6. Dependence of the coordinate system of the j-th part relative to the i-th part

In the production cycle of pipelines in coordinate systems preserving the Toi relation, it is possible to perform geometric matching of parts and assemblies with production equipment, in particular with traditional technological equipment for assembly of pipelines, using lodges, clamps and clamps for positioning parts.

Based on the analysis of the equipment design, we can conclude that for the lodgments made of sheet, that is, “flat” lodgments, the

LiX axis of the coordinate

system should be perpendicular to the plane of the lodgment and be perpendicular to the pipeline route. To simplify, we consider a special case of the lodgment for the i-th part of the pipeline, when the pipe is straight and the

LiX axis of the lodgment will be

perpendicular to the axis of the pipe.Similarly, you can write the expression of the connection to the coordinate system

for the tool tray, on which the detail of the pipeline is fixed before welding:,

,,

L Li i iL Li i iL Li i i

X X AY Y BZ Z C

where , ,L L Li i iX Y Z – the beginning of the coordinate system of the tool tray relative to

the i-th part; , ,L L Li i iA B C – the magnitude of the linear displacements of the origin of

the cradle relative to the i-th part.

Or in matrix form: , , L L L Li i i iT A B C

The relationship between the coordinate system of the i-th part and the lodgment is shown in figure 7.

Figure 7. Dependence of the coordinate system of the i-th part on the lodgmentThe considered transformations of the coordinate system of parts and lodgments

for brevity were presented without taking into account the rotation of the coordinate system. It is advisable to take into account the rotation angles of the coordinate system using Euler angles.

In addition to coordinating the coordinate systems of the part and equipment, it is required to coordinate with the coordinate system of the welding robot. The transition

95

matrix for the welding robot, taking into account the rotation of the coordinate system, can be written as

,0 1

R RR i ini

M TR

where RiM – rotation matrix for the coordinate system of the welding robot relative

to the coordinate system of the tooling; RiT – the transition matrix for the coordinate

system of the welding robot relative to the coordinate system of the snap.Now, taking into account the location of the robot relative to the snap-in, you can

record the recalculation of the welding points from the snap coordinate system to the robot coordinate system:

,Rij R ijn ni nW R W

where ijnW – welding point of the i-th and j-th parts in the tool coordinate system;

RijnW – welding point in the coordinate system of the robot.

Here the matrices ijnW and

RijnW have the form:

1

1

1

...

... .

...0 0 0 1

ij ij ijk

ij ij ijk

ij ij ijk

X X AY Y BZ Z C

The transition from the coordinate system of the tooling (tool tray) to the coordinate system of the welding robot and the welding points, which are set by the tool path, is shown in figure 8.

Figure 8. Dependence of tool coordinate systems, welding robot and welding point

96

When programming industrial robots, the problem of adjustment of coordinate systems between the robot, tooling and pipeline parts arises to ensure the required accuracy of the pipeline being welded due to errors in the positioning of tooling relative to the robot coordinate system, as well as errors in the installation of the part in the tooling (figure 9).

In this case, of particular importance is the ability to adjust the trajectory of the manipulator, for the implementation of which it is necessary to introduce correction factors.

Figure 9. Installation technological equipment offsetTo do this, we define the main types of positioning errors:1) linear deviations expressed in longitudinal and transverse displacement of

technological equipment;2) angular deviations between the position of the tooling relative to the

coordinate system of the welding robot.After finding the corrections, the coordinates will be recalculated in the control

program for welding according to the formulas:XN =XC + ΔXYN =YC + ΔYZN =ZC + ΔZ

when XN, YN, ZN – new coordinates as amended; XC, YC, ZC – calculated coordinates; ΔX, ΔY, ΔZ – correction factors containing linear and angular displacements.

Consider the implementation of this method in practice using a feedback system based on a laser tracker.

To implement this method, reference points (at least three) are installed on the base of the equipment and using a wireless contact probe and a laser tracker, the location of the robot relative to the pipeline fragment is found, but the location of the laser tracker relative to the welding manipulator is also determined by the reference points installed on it.

The constructive implementation of reference points may be different for different enterprises or departments depending on the required accuracy, working environment and other parameters.

The above reference mark is installed on the frame of the device.After determining the relative position of the robot and the details of the pipeline,

correction factors are introduced in the unitary enterprise.

97

The scheme of the laser tracker with a welding robot is shown in figure 10.

Figure 10. The scheme of linking the robot with technological equipment and piping using a laser tracker

6 Conclusion

The article presents skeleton models of parts and piping systems of aircraft systems. To describe skeleton models, a simplified representation of the ends of the welding joints using three points was chosen. The main feature of these models is the preservation of the coordinate relationship with the master geometry of the aircraft.

The presence of such a relationship can be used for adjustment coordinate systems at various stages of the aircraft life cycle. Calculation of errors in the assembly and the welding process can be performed using matrix transformations, for the implementation of which only three joint points are enough. The physical implementation of such coordination can be performed using laser trackers and introducing correction factors into the control program for the welding robot. This approach will increase the accuracy of the assembly of pipelines, which will reduce the number and need for modifications of the mounting units on the power elements of the aircraft glider frame.

98

Prospects of Development of Mining Machinery and Electrical Engineering, IPDME 2018; 2018.

10. Trushnikov V.E., Grishin M.V., Pavlov P.Yu. Matching a Welding Robot CoordinateSystem with Technological Equipment during the Assembly of Aircraft Pipes // International Conference on Innovations and Prospects of Development of Mining Machinery and Electrical Engineering, IPDME 2018; 2018.

References:1. Altfeld H.H. Commercial Aircraft Projects: Managing the Development of Highly

Complex Product Burlington: Ashgate, 2010.2. Amadori K. Geometry-Based Design Automation: Applied to Aircraft Modeling and

Optimization Linkoping University: Linkoping Studies In Science And Technology, Dissertations, No. 1418, 2012.

3. Haimes R., Drela M. On The Construction of Aircraft Conceptual Geometry for High-Fidelity Analysis and Design, In Proc. of the 50th AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition Nashville, USA: Aerospace Sciences Meeting, 2012.

4. Mart T., Cangelir C. Lessons Learned for Better Management of Master Geometry // InProc. of IFIP International Conference on Product Lifecycle Management. 2013. C. 712–721.

5. Mas F., Mendez J.L., Oliva M., Rios J. Engineering: an Airbus case study. // ProcediaEngineering. 2013. 63. C. 59–62.

6. Pardessus, T. Concurrent Engineering Development and Practices for Aircraft Design atAirbus In Proc. of the 24th ICAS meeting, 2004.

7. Rizzi A., Zhang M., Nagel B., Boehnke D.l. and Saquet P. Towards a UnifiedFramework using CPACS for Geometry Management in Aircraft Design, In Proc. of the 50th AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition Aerospace Sciences Meeting, Nashville, USA, 2012.

8. Sosnin P., Pavlov P. Precedent-Oriented Geometrical Modeling of the Parts andAssembly Units of an Aircraft Pipeline // IEEE 12th International Conference on Application of Information and Communication Technologies, Kazakhstan, Almaty, 17-19 October. 2018. C. 240–244.

9. Trushnikov V.E., Grishin M.V., Pavlov P.Yu. Wave Technologies for the Design ofProduction Tooling in Aircraft Industry // International Conference on Innovations and

99

Enhancing the ability to work with primitives through LWIQA

A. Ivasev

Ulyanovsk state technical university, Ulyanovsk, Russia e-mail: nevskei@yandex.ru

Abstract. In this paper, the author analyzes the importance of visual information and describes how to work with primitives in the diagram editor of the WIQA application.

1 Introduction

To ensure the coherence of potential issues and the development of solutions, people interest projects and analysts should be allowed to experiment with design decisions. [1] Thus, projects need tools to implement simulation of project fragments. An adult learns only 10 percent of the information read, 20 by ear, 30 visually, 40 by ear with visual reinforcement, 60 by oral discussion, 80 by self-searching and formulating the problem, and 90 percent by self-formulation and resolution of the problem. [2] However, the larger the flow of information we pass through ourselves, the more difficult it is to focus our attention and perceive it. The percentage of perception falls. And ultimately, the brain refuses to accept additional amounts of information. Therefore, the problem arises of structuring information and presenting it in a compressed format that will be clear to all interested parties, regardless of their level of knowledge and position. The article presents such tools, which are based on the extension of the pseudo-code language for working with the diagram editor and primitives on it. With their help, you can develop interfaces for experiments with programmable conceptual solutions in the WIQA tool environment.

2 The Importance of Visual Information

A healthy person has several sense organs through which they receive information: 1) Vision. With the help of the eyes, people distinguish colors, perceive visualinformation, which includes textual, numerical, and graphic; 2) Hearing. Ears help to perceive sound information - speech, music, sound signals,noise; 3) Smell. With the help of the nose, people get information about the smells of theenvironment. peace; 4) Taste. The taste buds of the tongue provide an opportunity to get information aboutwhat the object tastes like - bitter, sour, sweet, salty; 5) Touch. With your fingertips (or just skin), on the probe you can get informationabout the temperature of the object - whether it is hot or cold; the quality of its surface is smooth or rough.

Many studies determine the amount of information received by these senses. They differ in numbers, but they all say that more than 60% of the information is perceived by vision and therefore is the most trained body. So, according to the results of one of the studies, a person receives 65% of the information through vision, 20% through hearing, 7% through touch, 5% through smell and 3% through taste.

Figure 1. Perception of information,%

For this reason, a person faster processes information that was presented in a visualized form. Because of this, to increase the success of programming projects, various diagrams were invented, with the help of which they try to visualize tasks that must be solved. For the same reason, a chart editor was added to WIQA. With it, you can create various diagrams or prototypes of interfaces from primitives. And to add dynamics to the diagram, the ability to program diagrams is added, commands have been added to LWIQA that can change the state of the diagram editor. It is also possible to enter information from the diagram into the code and create an input data handler. For the convenience of the user and a better understanding of the program, it was decided to add the ability to set the identifier for the primitive. Thus, it is possible to associate a field with a variable in pseudo-code and allow editing and deleting specific primitives from the editor.

3 Adding primitives to the chart editor

Now there are several ways to add a primitive to the chart editor. The first is to select it in the editor and transfer it to the workspace.

101

Figure 2. Add primitive

After that, it becomes possible to set the name of the primitive, for this you need to click on the already placed primitive. On the right in the chart editor, a tab with the name “Shape” will appear among the tabs. If it is not selected, it will be necessary to click on it and we are presented with a list of primitive properties that we can change among them is “Name”, in which the identifier of the primitive is stored. By default it is empty and we can set it or change if it is already set. It is worth noting that there is no check for uniqueness, this is due to leaving the possibility of mass changes of similar elements and allowing changing a variable from several places if necessary.

Figure 3. Set primitive name

The second way to add a primitive to the editor is to use pseudo-code commands. There is no list of commands and their descriptions in the application. And to get acquainted with them, you need to add the primitives of interest to the editor in the first way and then transfer them to the pseudocode. To do this, select one of the vertices of the task tree into which you want to save the code for creating primitives. After that, select the file menu item, and select “Export Charts” in the pop-up menu, select “Pseudocode” in it. After that, a code will appear in the selected vertex, after which you can get the initial state of the diagram.

102

Figure 4. Save primitive in QA-model

4 Modifying and deleting primitives from the diagram editor

For an example, take one of the resulting vertices and see its text: Input ("MaintainOrientation" = "True", "Text" = "23", "AutoSize" = "False", "Width" = "60", "Height" = "28", "CharacterStyle" = "Normal", "ParagraphStyle" = "Title", "X" = "170", "Y" = "254", "Angle" = "0", "FillStyle" = "White", "Data" = "", "Name" = "Name22", "LineStyle" = "Normal") There are several properties that are responsible for the location of the primitive, its size and style display. We are interested in the “Name” property, which was added and in which the identifier of the primitive is stored, by which it can be found among all. If we look at this value in the lines, we will see that one primitive is called “Name12” and the other “Name22”. So if we want to change the text in the field that has the name “Name22”, we need to execute the command “DD_Update (" Name22 "," Text = _ ")”. And if we want to delete the button that has the name “Name12”, we need to execute the command “DD_Delete (" Name12 ")”. If we execute both commands, we get the following diagram:

103

Figure 5. Result

7 Conclusion

As was said at the beginning, a person receives most of the information through vision. Thus, it is easier for him to process the information provided in a visual form and he can immediately determine how much this information meets his expectations. Also, when people speak even the same language but have a different base of visual images under the same word, they can represent different things. So a person who has not been to the sea under the word “scallop” will understand a small comb when the other will represent the mollusk. Thus, it is important that in solving problems all of its participants had a common figurative base. This can be achieved by showing them the same diagrams, schemes, and prototypes. The chart editor allows you to create them. Our refinement allows us to add dynamics to the diagram, which should add understanding not only of static states but also how these states are related and under what conditions they change. Therefore, we believe that these changes will reduce the risk of uncoordinated and unconscious actions, which can cause failure in solving the task. References: 1. M.Romodin “Approach to prototyping project solutions interacting with data bases on the example of UAV” - INTERACTIVE SYSTEMS: Problems of Human - Computer Interaction – Collection of scientific papers. − Ulyanovsk: USTU, 2017. 2. Moiseeva S.O., Denisenko V.I. Problems of documentary support of the project // Economics and management of innovative technologies. 2012. No. 1 [Electronic resource]. URL: http://ekonomika.snauka.ru/2012/01/347 (accessed date: 02.10.2019). 3. How does a person perceive information? [Electronic resource]. URL: https://sites.google.com/site/ucebnyjproet/kak-celovek-vosprinimaet-informaciu (accessed: 10/02/2019)

104

Multi-agent Approach to Fill Project Ontology with the Help of Reasoning Texts

A. Kulikova

Ulyanovsk state technical university, Ulyanovsk, Russia e-mail: a.push1206@gmail.com

Abstract. This paper describes methods to process reasoning texts (particularly, question-and-answer units) in order to retrieve concepts and relations that could be added to a project ontology. The multi-agent approach to process such texts are presented and the algorithms developed for specific agents are described.

1 Introduction

Nowadays a great problem in the software development domain is the low success rate. According to the Standish Group research [1] in 1994 only 16% of software projects were successfully finished, now this number has risen up to 30 but, nevertheless, we still cannot call that satisfactory. Moreover, the complexity of the software systems being developed is constantly growing and that’s why the developers have to develop more new methods to provide control over the development process and to support understanding in the project team.

One of the possible ways to raise the quality rate is to start processing reasoning texts which are written by a designer while he or she deals with a project at its conceptual design phase. Such reasoning texts usually express complex project solutions and, therefore, can become the source of serious semantic errors which influence the whole design process in a negative way.

Using project ontologies, which contain all the important information related to a project, help to avoid such errors. A project ontology can be used to provide semantic control over the reasoning process and to gain a better understanding of a project task. More information on the ontological approach to dealing with software project tasks can be found in papers [2-6]

However, before that could be done, a designer has to fill the ontology with relevant concepts. Reasoning texts such as discourses or question-and-answer units become the source of information that can potentially be added to a project ontology. Thus, some tools should be developed in order to process such texts either and provide a designer with potential concepts and/or relations that could be added to a project ontology.

In this paper, we focus on natural language processing toolkit to retrieve potential concepts and relations from texts in Russian. The described tools are integrated into the WIQA instrumental environment (Working in Questions and Answers) [6]. This environment brings automation to the design process, particularly at its conceptual phase.

2 Software agents’ interaction within the concept retrieving process

Our toolkit includes some activities which can be fully automated. Moreover, during the research it turned out some activities can be carried out in parallel. Taking that into account, we decided to take advantage of the multi-agent approach while developing our toolkit. The agents perform such activities as splitting a text into wordforms, normalizing them, retrieving collocations, filtering out stopwords, discovering pairs of semantically related text fragments and others.

Figure 1 describes the sub-scheme of software agents’ interaction that deals with Russian texts. Rectangles contain artifacts used as input and output data for software agents, rounded rectangles contain software agents and circles contain auxiliary modules used by software agents.

Figure 1. Software agents’ interaction

Figure 1 shows that there are three parallel activities divided into subphases. A designer can use the output data of each subphase in order to control the overall processing. Moreover, the output data of Activitiy 1 (agents A-C) is used by Activity 2 (agents D-F), and the output data of Activity 2 is used by Activity 3 (agents G-H). So, once there is data to process, a corresponding agent is automatically launched: such a method helps to save time which is rather important because processing texts is usually time-consuming. Let us consider each activity in a more detailed way.

Activity 1. At first, a text is split into wordforms by spaces and punctuation marks. After that, the wordforms are normalized with the help of a morphological analyzer. We use the Yandex MyStem tool [7] to process text in Russian since it turned out to be

106

faster than many others and it can be freely used for research purposes. Finally, the normalized wordforms undergo a filter which removes stopwords (words that do not contain any significant semantic information) as well as words that are already present in the project ontology as concepts. A designer can use List 1 to fill the project ontology or he/she can proceed to the results of Activity 2.

Activity 2. The “task” of Activity 2 is to retrieve collocations from the same text. At first, all possible collocations are extracted. We studied several philological papers on what is collocation in the Russian language and in which speech parts it can be expressed (see Table 1) and used them to form a set of rules that help to retrieve collocations.

Finally, collocations also undergo a filter which removes from the list of collocations the ones that do not contain any noun. We developed that filter algorithm based on the hypothesis which stated that an ontological concept should always contain a noun. This hypothesis confirmed during the experiment when we extracted concepts manually from a set of texts and added the ones we considered to be relevant to the project ontology.

Table 1. Syntactical models of collocations in the Russian language

Model Example Verb + Noun / Noun + Verb купить хлеба, жалеть людей Verb + Pronoun / Pronoun + Verb понравиться ей Verb + Preposition + Noun / Preposition + Noun + Verb

подойти к причалу, сесть на землю

Verb + Preposition + Pronoun / Preposition + Pronoun + Verb

обратиться к нему

Verb + Participle / Participle + Verb сидеть задумавшись Verb + Infinitive / Infinitive + Verb просить приехать, предложить

отдохнуть Noun + Noun порог дома, речь президента Noun + Preposition + Noun / Preposition + Noun + Noun

стол под деревом, боль от ушиба

Noun + Adjective / Adjective + Noun железная кровать, полезное дело полный смущения, покорный судьбе

Pronoun adjective + Noun / Noun + Pronoun adjective

моя книга, ваша семья

Noun + Numeral / Numeral + Noun второй день, шестая рота Participle + Noun / Noun + Participle прочитанная книга,

выглаженная рубаха Noun + Adverb / Adverb + Noun удар наотмашь, прогулка

пешком Noun + Infinitive / Infinitive + Noun намерение возвратиться,

умение рассказывать Adjective + Preposition + Noun / Preposition + Noun + Adjective

черный от загара, смелый от рождения

107

Adjective + Pronoun / Pronoun + Adjective нужный нам, приятный вам Adjective + Preposition + Pronoun / Preposition + Pronoun + Adjective

близкий для себя, трудный для нас

Adjective + Adverb / Adverb + Adjective по-летнему зеленый, дружески заботливый

Adjective + Infinitive / Infinitive + Adjective готовый бороться, способный любить

Numeral + Adjective / Adjective + Numeral / Numeral + Adjective / Adjective + Numeral

третий от конца, первый из трех, пятый из пассажиров, две книги, трое в шинелях

Adverb + Adverb очень ловко, весьма искусно, совершенно одинаково

Noun + Preposition + Adverb / Preposition + Adverb + Noun

смешно до слез, далеко от друзей

Activity 3. Retrieving semantic relations is a very complex task which interests linguists from all over the world and which has not yet been solved. However, for the purposes of our research, we do not need very precise results and, thus, we decided to apply a very simple approach to retrieve semantic relations, i.e. using a set of tags (or semantic markers) which state that there are relations of a certain type in a specific sentence.

According to some philological studies, semantic relations are most explicitly expressed by verbs and unions. So, we used the synsets of the Russian thesaurus RuWordNet to form a set of tags for each relation type that we wanted to retrieve and agreed that each sentence can be considered as follows:

• [Main phrase] <Related phrase tag> [Related phrase] • [Related phrase] <Main phrase tag> [Main phrase]

For example, the sentence “Текст состоит из предложений”, firstly, is considered to contain the relation of a part-and-whole type since there is a verb related phrase tag “состоять”; secondly, the first part of the sentence (before the tag) is considered to be the whole and the second part of the sentence (after the tag) is considered to be the part. So, after processing the text we get the following result:

• предложение <is part of> текст. After such semantically related pairs of phrases are retrieved, Agent H uses the list

of potential concepts formed during Activities 1 and 2 and checks if any pairs of phrases contain these concepts. If they do, the agent considers them as semantically related.

3 Conclusion

After making an attempt to apply the ontological approach to a real software project, we found out that filling the ontology manually is rather time-consuming. Moreover, some concepts and relation can be overlooked when added manually.

That’s why using a toolkit which automates this process has many advantages and its development contributes to raising the success rate of software project development.

108

References: 1. Standish Group 2015 Chaos Report - Q&A with Jennifer Lynch [Электронный ресурс]

– Режим доступа: https://www.infoq.com/articles/standish-chaos-2015. 2. Sosnin P., Kulikova A., Shumilov S. (2019) Architectural Approach to Ontological

Maintenance of Solving the Project Tasks in Conceptual Designing a Software Intensive System. In: Kravets A., Groumpos P., Shcherbakov M., Kultsova M. (eds) Creativity in Intelligent Technologies and Data Science. CIT&DS 2019. Communications in Computer and Information Science, vol 1084. Springer, Cham.

3. P. Sosnin, A. Kulikova. Ontology-Based Way of Formulating the Statements of Project Tasks in Designing a System with Software. Proceedings of the 18th International Conference on Computational Science and Applications (ICCSA 2018) – p. 25-30.

4. Ontology-Based Specifications of Concerns in Architectural Modeling of a Software Intensive System. 2018 26th Telecommunications Forum (TELFOR). Proceedings of Papers. Belgrade, Serbia, November, 20-21, 2018. – p. 843-845.

5. Sosnin P., Pushkareva A., Negoda V. Ontological Support of Design Thinking in Developments of Software Intensive Systems. In: Abraham A., Kovalev S., Tarassov V., Snasel V., Vasileva M., Sukhanov A. (eds) Proceedings of the Second International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’17). IITI 2017. Advances in Intelligent Systems and Computing, vol 679. Springer, Cham – p. 159-168.

6. Sosnin, P.: Experience-Based Human-Computer Interactions: Emerging Research and Opportunities, IGI-Global, (2017).

7. Ilya Segalovich. A fast morphological algorithm with unknown word guessing induced by a dictionary for a web search engine. [Электронный ресурс] – Режим доступа: https://cache-mskstoredata03.cdn.yandex.net/download.yandex.ru/company/iseg-las-vegas.pdf.

109

Научное электронное издание

Interactive Systems Workshop 2019

Сборник научных трудов

Ответственный за выпуск: П.И. Соснин

Объём: 8,02 МБ

Дата подписания к использованию: 17.12.2019.