Gagarin russian-language interactive humanoid

Imagine a world, when all monotonous labor is performed by robots, and humans are free to unleash their creative potential. Currently, there are many researches oriented for realisation of this future in industry, at factories and plants. In other hand, there are wide possibilities for robotization in more social directions, which we are covering by “Gagarin” research project. We are going to develop social-oriented robot, who is available to communicate with humans about concrete topics, can adjust its behavior according to interlocutor’s emotions and, moreover, communicate using non-verbal ways (facial emotions). In addition, this robot can became cooperative one, using all possible data and skills, which we are planning to develop.

Qualitative Trajectory Calculus (QTC)

The main objectives of the project are two-fold:

  • Extend the so called “inverse problem” of QTC, providing a more generic solution then in [1], optimizing the usage of grisds, and extending to n-points and 3-D inverse QTC

  • Apply n-D QTC to interesting longitudinal socio-cultural data of nations

In more detail:

  • Full diagrams for the region characterizations and connectivity for the generic solution of 2-point and 3-point QTC. Two-three examples of discrete point choice grids, together with the justification of their choice. A set of criteria for selecting discrete point choice grids (mix of computational with functional criteria). Algorithms implemented in matlab for deterministic (grid based) as well as stochastic (distribution-based) inverse QTC trajectory generation, for 2-, 3-, as well as n-point cases. Exploration of the small-step local linearization-based solution. Exploration and justification of different distribution choices covering the regions for the stochastic algorithms. Investigation and proposal of solutions for the uncontrollable “expansion” and “shrinking” problems. Time complexity study of solutions, and comparison against dummy algorithms. Investigation of the use of partial pre-computed inversion sequences. Creation of videos for generated trajectories for some examples. Proof of inversibility of trajectories (i.e. that when they are fed back to a QTC encoder we end up with the same original sequence).
  • Description of the n-D generalization of 3-D QTC devised (both devised by N. Mavridis) [write down equations clearly]. Implementation of the n-D QTC encoder in matlab, starting with 4-D. Very initial description of the two candidate datasets, including justified proposals for which subset of dimensions (socio-economical indicators of nations: Gallup, and the Economic Complexity Index) to be used for the second part of the thesis.

EEG BMI for Robot Grasping

The main objectives of the project are the following:

  • Perform pre-processing and classification of 128-electrode EEG data collected through a suitable experiment for a 4-category motion imagination task, using reshaping into appropriate tensors, and appropriate feature extraction and classification methods.

  • Furthermore, investigate quasi-real-time online classification.

  • Use the classifier outputs to drive a simulated robot performing 4 predetermined actions

  • Use the classifier outputs to drive a physical robot performing 4 predetermined actions

  • Use machine vision methods for classifying target objects illuminated with low frequency flcker (5-20Hz).

  • Use pre-processing and classification for object selection through the EEG signal

  • Combine steps f) and d) in order to generate object-directed actions through a simulated and real robot.

  • Investigate various ways to move across the continuum between autonomous grasping and continuously-human-controlled servoing

Music generation using GANNets

The main objectives of the project are:

  • Devise appropriate data sets and ways to encode human musical compositions from a selected variety of genres, with increasing complexity (monophonic to polyphonic, pentatonic to folk music to art music etc.)

  • Create a generative adversarial network, and fine tune its parameters, in order to generate music that converges statistically to the properties of selected chosen datasets.

  • Investigate the subjective human-likeness of the artificially generated music through appropriate human experiments (turing-test like, similarity space-based, or otherwise)

  • Iteratively refine a)-c)

  • Investigate the hierarchical structure of musical compositions (note, motif, chord, tonic center changes and modulations) (as well as rhythm-melody and harmony-counterpoint relations) and its ramifications as portrayed in deep learning layers.

  • Investigate the differences between nature-generated sounds and musically-composed sounds in light of their artificial generation

  • Investigate all of the other important resultant questions associated with the above, including data set size effects, genre effects, inter-human variations, etc.

Robot collision recognition, localization, and handling

The main objectives of the project are:

  • Create a typology of different types of collisions, between robot-(self robot body), robot-(other robot), robot-human, robot-object, robot-(ground or wall etc). Create further categorizations, in terms of static vs dynamic objects, point contacts and sliding moves, rigitv vs. soft collisions etc. Be informed by an appropriate bibliographical study of robot collisions with at least 30 relevant papers.

  • Implement functionality for the KUKA robot enbling: i) going through arbitrary trajectories, given either in joint angle timeseries mode or in end-effector position-pose mode, ii) receiving (at a rate of 20Hz or more) joint angle sequences (from angular encoders) and torque measurements

  • Implement forward and inverse kinematics models for the robot

  • Design appropriate training sets: i.e. chose at least 3 categories of collisions from the typology created in a), and create training sets which consist of 10 events each (in the pilot) and 100 events each (in the final version). The training sets should contain: desired trajectory, actual encoder timeseries values, actual torque timeseries values, and also a dynamic 3D model of the robot and object acquired through the 3D scanning laser of the lab, providing accurate ground truth for later evaluation of our automated estimation/classification methods. The duration of each training set sample should be 10 seconds or so. Careful choices must be made regarding the chosen trajectories, colliding objects, points of collision, sliding mode etc.

  • Investigate the in-built protection mechanisms of the KUKA robot (detection of tracking error continuous increase - i.e. difference between actual and desired joint angle increasing), current sensing, thermal sensing etc.

  • Create automated classification/estimation methods providing answers to the following questions: Q1] Did a collision take place, and if so, when and for how long? Q2] What type of collision happened (and how many meaningful types of collision we can define?) Q3] Where did the collision happen? Q4] How did the collision evolve over time, in terms of contact point and forces, and in terms of the positions/poses of both the robot and the colliding object? Q5] How can we design appropriate reaction strategies from the robot, and what are objective functions/metrics for their effectiveness? Propose, implement, test, evaluate. Q6] How can we estimate the mass, stifness, geometry, and motion of the colliding object? Q7] When is a full robot forward kinematics model required, and how much can one answer just by observing joint angle deviations from desired values over time? Q8] How can special trajectories by generated in order to turn the robot into a “haptic sensing” mode, and how can 3D laser-based ground truth help in this direction. [A full investigation with clear articulation of the research questions, of assumptions, of parameter settings, and with graphs and analysis of results is expected]

Generation of possible genealogical trees

The main objectives of the project are:

  • Devise appropriate objects and methods in order to encode human genealogical trees, primarily encoding parent-child relations, but also producing more complex relations (grandparents, siblings, half-siblings, and all other valid types up to fifth-order cousins). The trees should also include gender, as well as age of birth and death. Methods for generating listings of all existing relations and for querying the validity of specific relations should be included. Also, provisions for unknown/incomplete nodes should be made.

  • Devise methods for generating realistic random genealogical trees, that follow culture-, date-, and other context-specific parameters (for example, distribution of number of children and ages of parents during birth etc.). Enlist carefully all assumptions and parameters for the implemented tree generation methods (for example, assumptions regarding intermixing etc). Find appropriate bibliographical sources for fine-tuning such parameters, and try to compare with existing online database sources. Evaluate the methods and give quantitative test results in appropriate graphs.

  • Devise methods for importing and exporting into standard file formats such as GEDcom

  • Devise methods for randomly deleting a number of nodes in order to create partial trees that structurally and statistically resemble those found on human DNA geneological databases such as GEDmatch, i.e. where often there exists information for a few close relatives, and then just a few third- and beyond-level cousins only. Devise ways to measure the “GEDmatch-likeness” of the constructed trees. Evaluate the methods and give quantitative test results in appropriate graphs.

  • Examine the effect of a number of GEDmatch-available standard parameters (total matching segments in centiMorgans, largest matching segment etc.), on the distribution of potential relations (from the enumeration of relations constructed in a)).

  • Most importantly, given a matrix of the standard overlap measure parameters of e), generate potential genealogical trees, and try to estimate their probability. You can start by manually entering trees and checking their consistency with the overlap measure matrices.

  • Include further constraints in the genaration of trees, such as: known relations, Y- and mt- haplogroups, etc.

  • Create a convenient user interface for exercising all of the above, and for connecting with file formats and copied-pasted information for the major related websites (GEDmatch etc.)

Werewolf video analysis

The main objectives of the project are:

  • Collect and understand the existing research regarding the mathematical, psychological, and sociological aspects of the werewolf game

  • Devise, implement, and test methods for automatically processing werewolf videos from youtube, including: segmentation and extraction of windows, quantification of movement in windows, detection of which window is being focused as the main window, detection of day and night, detection of departure of players.

  • By using the methods of b), create (to a large extent automatically) transcripts of event sequences for specific werewolf game episodes

  • Investigate the viability and devise methods for classification of players into their game group categories, on the basis of video content as well as other extractable features

  • Extract interesting statistics and analytics regarding werewolf game videos in youtube or otherwise

  • Investigate the viability and devise methods for automatic recognition of other major events during games

  • Utilize youtube ASR or other speech rec methods, as well as surface-level speech characteristics (speech durations, speech overlap, pitch etc.), in combination with d)

  • Create a nice user interface for the above functionalities, and evaluate it

Smart Buildings using Distributed Microservices

This project is complementary to our existing “Jolie Good Buildings” micro-services-based smart buildings infrastructure project. The main objectives of the project are:

  • Create an android application, with an appropriately designed user interface, that can:

i- Connect to the nearest sensor node(s) through Bluetooth

ii- Connect to the central server through wifi or otherwise

ii- Display sensor data in real-time (as well as histories)

iv- Get feedback from users (when the user wants or when the system asks) regarding comfort as well as other subjective parameters (is it too dark? Too cold? Are you tired? Etc)

v- Localize the user’s phone using techniques such as those described in the Microsoft “Indoor Localization” competition 2016. The existing static (fixed-position) nodes can help enhance the localization

vi- Use the phone as a normal sensor node, yet mobile (in contrast to the classic fixed-position sensor nodes of the rest of the system), providing position, owner, accelerometer, GPS, and other useful data

Create two-three scenarios of usage involving/demonstrating/testing all of the above

  • Create two-three scenarios of usage involving/demonstrating/testing all of the above

  • Create a prototype application utilizing a) but also our whole microservices-based Jolie Good Buildings Infrastructure that can be used for hospitality to Innopolis visitors.

  • Evaluate the usability of the application through an appropriately designed experimental study with at least 20 subjects and refine the application on the basis of the results.

Cross-site humanoid telepresence

The main objectives of the project are:

  • Create inverse kinematics code (or otherwise uniformly interface (through ROS or sockets) with existing code) towards moving the end-effectors (hands) of both humanoids (H1: Hubo and H2: AR601) to desired pose/position combinations, enabling quick tracking of intermediate points

  • Create code (or otherwise uniformly interface (through ROS or sockets) with existing code) towards sending streams of human hand positions (and potentially poses) for both hands of a human observed by an appropriately placed kinect (or other similar RGB-D sensor)

  • Demonstrate real-time copying of human hand positions by both robots, with the kinect and human in the same room as the robot. Fine tune parameters for starting/ stopping/ stepping motions toward appropriate smoothness. Produce video of results.

  • Create code and demonstrate real-time copying of human hand positions by both robots, with the kinect&human placed across sites (i.e. human in Tatarstan, robot in Korea and vice-versa) Fine tune parameters for starting/stopping/stepping motions toward appropriate smoothness, and towards dealing with connection/network delays and imperfections.

  • Create (or otherwise utilize existing) code for voice and video communication, utilizing microphones and cameras placed inside/on the humanoid robots, with audio output in speakers of the humanoids (possibly with lip-syncing if a human-like face is used), and also demonstrate transfer of video to head-mounted display.

  • Experiment with two predictive models compensating for network delays, and illustrate quantitative results of the experiments

  • Extend the telepresence (both from the motor control side as well as from the RGB-D sensing side) in order to address more than just the hand end-effector position, but to also include feet movements.

  • Extend the telepresence (both from the motor control side as well as from the RGB-D sensing side) to also include more detailed joint-to-joint mappings (following an appropriate correspondence choice)

  • Create an appropriate experimental design for an interesting human-robot interaction experiment using symmetric telepresence (potentially across multiple sites with a mix of humans and robots), run the experiment with at least 20 human subjects, gather and analyze results (including statistical significance of hypothesis, models and statistics of parameters etc.), and produce appropriate report.

Innopolis RoboWaiter

The objectives of this project are:

  • Create mechanical extension for our Pioneer mobile robot, to emulate the Peoplebot and enable screen-based head-height human-robot interaction, and handling of trays for serving coffee/food.

  • By building upon ARIA/ANRL or otherwise, enable floor-level point-to-point navigation for fixed landmarks, and signaling for opening doors of offices

  • Demonstrate delivery of trays on the same floor, with graceful recovery from possible erroneous situations.

  • Extend to multiple floors through the use of the Innopolis elevators, originally with human assistance, and ideally autonomously

  • Interconnection to our smart buildings infrastructure and the Gagarin conversational humanoid for joint functionality

Vision for UAVs/Drones

The objectives of this project are:

  • Enable UAV-based car tracking for a set of increasingly more difficult scenarios, originally working with human-controlled UAV’s and their captured videos, and later moving to autonomous car following, for scenarios potentially involving crowded traffic, intersections, occlusions, illumination changes etc.

  • Interconnect the UAV  our smart buildings infrastructure, the Gagarin conversational humanoid, and our pioneer mobile robot, for joint functionality

Small-humanoid stair climbing

The objectives of this project are:

  • Enable small humanoids (such as the Nao) to climb stairs which are much larger than their foot span, through special sequences of body motions

  • Derive the action sequences through a variety of methods including trial-and-error, evolutionary computation, reinforcement learning. Use both simulation as well as real-world physical experiments, and investigate the relation between the two

Сайт находится в технической разработке