Mentions légales du service

Skip to content
Snippets Groups Projects
Commit d31e3e6c authored by Severin Lemaignan's avatar Severin Lemaignan
Browse files

updated spring-architecture.md

parent b992be95
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 447 additions and 340 deletions
spring-architecture.md
+ 447
− 340
View file @ d31e3e6c
# SPRING architecture # SPRING architecture
**Version:** 0.3.2 **Version:** 0.5.0
EU H2020 SPRING architecture EU H2020 SPRING architecture
...@@ -8,46 +8,51 @@ EU H2020 SPRING architecture ...@@ -8,46 +8,51 @@ EU H2020 SPRING architecture
| **Node** | **id** | **Partner** | **Status** | | **Node** | **id** | **Partner** | **Status** |
|----------|--------|-------------|------------| |----------|--------|-------------|------------|
| [ASR](#asr) | asr | BIU | mock-up |
| [Depth estimation from monocular](#depthestimationfrommonocular) | depthestimationfrommonocular | UNITN | mock-up |
| [Multi-people body tracker](#multipeoplebodytracker) | multipeoplebodytracker | INRIA | mock-up |
| [hri_fullbody](#hri_fullbody) | hri_fullbody | PAL | mock-up |
| [Sound source localisation](#soundsourcelocalisation) | soundsourcelocalisation | BIU | mock-up |
| [RGB head](#rgbhead) | rgbhead | PAL | mock-up |
| [Activity reco](#activityreco) | activityreco | UNITN | mock-up |
| [Face detection](#facedetection) | facedetection | UNITN | mock-up |
| [ spring_msgs](#spring_msgs) | spring_msgs | Other | released (version 0.0.2) (dependency) |
| [Robot GUI](#robotgui) | robotgui | ERM | mock-up |
| [respeaker_ros](#respeaker_ros) | respeaker_ros | PAL | released (version master) |
| [Person re-identification](#personreidentification) | personreidentification | UNITN | mock-up | | [Person re-identification](#personreidentification) | personreidentification | UNITN | mock-up |
| [ORB SLAM](#orbslam) | orbslam | PAL | mock-up | | [Non-verbal behaviours](#nonverbalbehaviours) | nonverbalbehaviours | UNITN | mock-up |
| [torso_rgbd_camera](#torso_rgbd_camera) | torso_rgbd_camera | PAL | mock-up |
| [ InLoc Server](#inlocserver) | inlocserver | CVUT | mock-up |
| [ spring_msgs](#spring_msgs) | spring_msgs | PAL | released (version 0.0.2) |
| [User attention estimation](#userattentionestimation) | userattentionestimation | UNITN | mock-up |
| [ interaction_manager_msgs](#interaction_manager_msgs) | interaction_manager_msgs | HWU | released (version spring_dev) (dependency) |
| [interaction_manager](#interaction_manager) | interaction_manager | HWU | released (version spring_dev) |
| [ hri_msgs](#hri_msgs) | hri_msgs | PAL | released (version 0.1.1) (dependency) |
| [Object detection/identification/localisation](#objectdetectionidentificationlocalisation) | objectdetectionidentificationlocalisation | CVUT | mock-up |
| [Voice speech matching](#voicespeechmatching) | voicespeechmatching | BIU | mock-up |
| [Speaker identification](#speakeridentification) | speakeridentification | BIU | mock-up |
| [mask_detector](#mask_detector) | mask_detector | UNITN | released (version master) |
| [hri_person_manager](#hri_person_manager) | hri_person_manager | PAL | released (version master) |
| [soft_biometrics_estimator](#soft_biometrics_estimator) | soft_biometrics_estimator | UNITN | released (version master) |
| [ROS mediapipe/openpose](#rosmediapipeopenpose) | rosmediapipeopenpose | INRIA | mock-up |
| [People 3D tracker](#people3dtracker) | people3dtracker | INRIA | mock-up | | [People 3D tracker](#people3dtracker) | people3dtracker | INRIA | mock-up |
| [Fisheye](#fisheye) | fisheye | PAL | mock-up |
| [F-formation](#fformation) | fformation | UNITN | mock-up | | [F-formation](#fformation) | fformation | UNITN | mock-up |
| [ robot_behaviour_msgs](#robot_behaviour_msgs) | robot_behaviour_msgs | HWU | released (version spring_dev) (dependency) |
| [respeaker_ros](#respeaker_ros) | respeaker_ros | PAL | released (version master) |
| [fisheye](#fisheye) | fisheye | PAL | mock-up |
| [dialogue arbiter](#dialoguearbiter) | dialoguearbiter | HWU | released (version spring_dev) |
| [Face detection](#facedetection) | facedetection | UNITN | mock-up |
| [Depth estimation from monocular](#depthestimationfrommonocular) | depthestimationfrommonocular | UNITN | mock-up |
| [Robot functional layer](#robotfunctionallayer) | robotfunctionallayer | PAL | mock-up | | [Robot functional layer](#robotfunctionallayer) | robotfunctionallayer | PAL | mock-up |
| [robot_behavior](#robot_behavior) | robot_behavior | INRIA | mock-up | | [ audio_msgs](#audio_msgs) | audio_msgs | HWU | released (version spring_dev) (dependency) |
| [User attention estimation](#userattentionestimation) | userattentionestimation | UNITN | mock-up | | [FairMOT Multi-people body tracker](#fairmotmultipeoplebodytracker) | fairmotmultipeoplebodytracker | INRIA | released (version devel) |
| [RGB-D + camera_info](#rgbdcamera_info) | rgbdcamera_info | PAL | mock-up | | [MoDE](#mode) | mode | BIU | released (version BIU_dev) |
| [Non-verbal behaviours](#nonverbalbehaviours) | nonverbalbehaviours | UNITN | mock-up | | [Speech synthesis](#speechsynthesis) | speechsynthesis | PAL | mock-up |
| [Speaker identification](#speakeridentification) | speakeridentification | BIU | mock-up | | [ social_scene_msgs](#social_scene_msgs) | social_scene_msgs | HWU | released (version spring_dev) (dependency) |
| [Mask detection](#maskdetection) | maskdetection | UNITN | mock-up | | [User visual focus](#uservisualfocus) | uservisualfocus | UNITN | mock-up |
| [ORB SLAM](#orbslam) | orbslam | PAL | mock-up |
| [robot_behavior](#robot_behavior) | robot_behavior | INRIA | released (version devel) |
| [Semantic mapping](#semanticmapping) | semanticmapping | CVUT | mock-up | | [Semantic mapping](#semanticmapping) | semanticmapping | CVUT | mock-up |
| [Speech analysis](#speechanalysis) | speechanalysis | BIU | mock-up | | [social_scene_context_understanding](#social_scene_context_understanding) | social_scene_context_understanding | HWU | released (version spring_dev) |
| [Soft biometrics](#softbiometrics) | softbiometrics | UNITN | mock-up | | [Robot GUI](#robotgui) | robotgui | ERM | mock-up |
| [ros_openpose](#ros_openpose) | ros_openpose | UNITN | released (version 0.0.1) |
| [Activity reco](#activityreco) | activityreco | UNITN | mock-up |
| [ wp4_msgs](#wp4_msgs) | wp4_msgs | UNITN | released (version master) (dependency) |
| [robot_behaviour_plan_actions](#robot_behaviour_plan_actions) | robot_behaviour_plan_actions | HWU | released (version spring_dev) |
| [Multi-person audio tracking](#multipersonaudiotracking) | multipersonaudiotracking | BIU | mock-up |
| [InLoc-ROS](#inlocros) | inlocros | CVUT | mock-up |
| [google_asr](#google_asr) | google_asr | BIU | released (version BIU_dev) |
| [Occupancy map](#occupancymap) | occupancymap | CVUT | mock-up | | [Occupancy map](#occupancymap) | occupancymap | CVUT | mock-up |
| [dialogue arbiter](#dialoguearbiter) | dialoguearbiter | HWU | mock-up | | [raspicam](#raspicam) | raspicam | PAL | mock-up |
| [plan_actions](#plan_actions) | plan_actions | HWU | mock-up |
| [User visual focus](#uservisualfocus) | uservisualfocus | UNITN | mock-up |
| [ hri_msgs](#hri_msgs) | hri_msgs | Other | released (version 0.1.1) (dependency) |
| [Speech synthesis](#speechsynthesis) | speechsynthesis | PAL | mock-up |
| [Scene understanding](#sceneunderstanding) | sceneunderstanding | HWU | mock-up |
| [ROS openpose](#rosopenpose) | rosopenpose | INRIA | mock-up |
| [Visual SLAM + 3D map](#visualslam3dmap) | visualslam3dmap | CVUT | mock-up |
| [Object detection/identification/localisation](#objectdetectionidentificationlocalisation) | objectdetectionidentificationlocalisation | CVUT | mock-up |
| [Body 2D pose](#body2dpose) | body2dpose | UNITN | mock-up |
| [interaction_manager](#interaction_manager) | interaction_manager | HWU | mock-up |
| [Person manager](#personmanager) | personmanager | INRIA | mock-up |
| [Voice speech matching](#voicespeechmatching) | voicespeechmatching | BIU | mock-up |
## Detailed description ## Detailed description
...@@ -55,11 +60,11 @@ EU H2020 SPRING architecture ...@@ -55,11 +60,11 @@ EU H2020 SPRING architecture
--- ---
### asr ### personreidentification
Node *ASR* (id: `asr`) is overseen by BIU. Node *Person re-identification* (id: `personreidentification`) is overseen by UNITN.
REPO [TBC]: https://gitlab.inria.fr/spring/wp5_spoken_conversations/asr MOCK: Person re-identification is...
#### Status #### Status
...@@ -67,21 +72,21 @@ REPO [TBC]: https://gitlab.inria.fr/spring/wp5_spoken_conversations/asr ...@@ -67,21 +72,21 @@ REPO [TBC]: https://gitlab.inria.fr/spring/wp5_spoken_conversations/asr
#### Inputs/outputs #### Inputs/outputs
- Input: `/audio/postprocess_audio_streams [audio_common_msgs/AudioData]` (topic) - Input: `/h/f/*/roi [sensor_msgs/RegionOfInterest]` (topic)
- Output: `/audio/speech_streams -- array of std_msgs/String` (undefined) - Output: `person_id_candidate [hri_msgs/IdsMatch]` (undefined)
#### Dependencies #### Dependencies
- `audio_common_msgs/AudioData`
- `std_msgs/Empty` - `std_msgs/Empty`
- `sensor_msgs/RegionOfInterest`
--- ---
### depthestimationfrommonocular ### nonverbalbehaviours
Node *Depth estimation from monocular* (id: `depthestimationfrommonocular`) is overseen by UNITN. Node *Non-verbal behaviours* (id: `nonverbalbehaviours`) is overseen by UNITN.
...@@ -91,24 +96,25 @@ Node *Depth estimation from monocular* (id: `depthestimationfrommonocular`) is o ...@@ -91,24 +96,25 @@ Node *Depth estimation from monocular* (id: `depthestimationfrommonocular`) is o
#### Inputs/outputs #### Inputs/outputs
- Input: `input` (undefined) - Input: `/h/f/*/roi [hri_msgs/RegionOfInterest]` (topic)
- Input: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
- Output: `depth` (undefined) - Output: `/h/f/*/expression [hri_msgs/Expression]` (topic)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `hri_msgs/RegionOfInterest`
- `hri_msgs/Expression`
- `audio_common_msgs/AudioData`
--- ---
### multipeoplebodytracker ### torso_rgbd_camera
Node *Multi-people body tracker* (id: `multipeoplebodytracker`) is overseen by INRIA. Node *torso_rgbd_camera* (id: `torso_rgbd_camera`) is overseen by PAL.
REPO (Docker): https://gitlab.inria.fr/spring/wp3_av_perception/docker-tracking
This code is primarily developed at INRIA by Luis Gomez Camara.
#### Status #### Status
...@@ -116,21 +122,19 @@ This code is primarily developed at INRIA by Luis Gomez Camara. ...@@ -116,21 +122,19 @@ This code is primarily developed at INRIA by Luis Gomez Camara.
#### Inputs/outputs #### Inputs/outputs
- Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic)
- Output: `/h/b/*/roi [hri_msgs/RegionOfInterestStamped]` (topic) - Output: `/torso_rgbd_camera/color/image_raw [sensor_msgs/Image]` (topic)
#### Dependencies #### Dependencies
- `sensor_msgs/Image` - `sensor_msgs/Image`
- `hri_msgs/RegionOfInterestStamped`
--- ---
### hri_fullbody ### inlocserver
Node *hri_fullbody* (id: `hri_fullbody`) is overseen by PAL. Node * InLoc Server* (id: `inlocserver`) is overseen by CVUT.
...@@ -140,54 +144,37 @@ Node *hri_fullbody* (id: `hri_fullbody`) is overseen by PAL. ...@@ -140,54 +144,37 @@ Node *hri_fullbody* (id: `hri_fullbody`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic)
- Input: `/h/b/*/roi [hri_msgs/RegionOfInterestStamped]` (topic)
- Output: `tf: /full_body` (tf)
- Output: `/h/b/*/skeleton2d [hri_msgs/Skeleton2D]` (topic)
- Output: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic)
#### Dependencies #### Dependencies
- `tf/transform_broadcaster`
- `hri_msgs/Skeleton2D`
- `sensor_msgs/Image`
- `hri_msgs/RegionOfInterestStamped`
--- ---
### soundsourcelocalisation ### spring_msgs
Node *Sound source localisation* (id: `soundsourcelocalisation`) is overseen by BIU.
Node * spring_msgs* (id: `spring_msgs`) is overseen by PAL.
REPO:git@gitlab.inria.fr:spring/wp7_ari/spring_msgs.git NOT EXECUTABLE
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: 0.0.2**
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
- Output: `source angle` (undefined)
- Output: `tracking information` (undefined)
- Output: `tf: /voice_id` (tf)
- Output: `output` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty`
- `tf/transform_broadcaster`
- `audio_common_msgs/AudioData`
--- ---
### rgbhead ### userattentionestimation
Node *RGB head* (id: `rgbhead`) is overseen by PAL. Node *User attention estimation* (id: `userattentionestimation`) is overseen by UNITN.
...@@ -197,43 +184,57 @@ Node *RGB head* (id: `rgbhead`) is overseen by PAL. ...@@ -197,43 +184,57 @@ Node *RGB head* (id: `rgbhead`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `TF (faces)` (undefined)
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterest]` (topic)
- Output: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic) - Output: `tf: /face_id_gaze` (tf)
- Output: `x,y + attention heatmap` (undefined)
#### Dependencies #### Dependencies
- `sensor_msgs/Image` - `tf/transform_broadcaster`
- `std_msgs/Empty`
- `hri_msgs/RegionOfInterest`
--- ---
### activityreco ### interaction_manager
Node *Activity reco* (id: `activityreco`) is overseen by UNITN.
Node *interaction_manager* (id: `interaction_manager`) is overseen by HWU.
REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/interaction.git
SUBFOLDER:interaction_manager
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: spring_dev**
#### Inputs/outputs #### Inputs/outputs
- Input: `gaze direction` (undefined) - Input: `semantic scene description` (undefined)
- Input: `TF (bodies)` (undefined) - Input: `/h/p/*/softbiometrics [hri_msgs/Softbiometrics]` (topic)
- Input: `TF` (undefined)
- Input: `robot state` (undefined)
- Input: `dialogue state` (undefined)
- Output: `[?] output` (undefined) - Output: `verbal command` (undefined)
- Output: `nav goals` (undefined)
- Output: `who to look at` (undefined)
- Output: `active personID` (undefined)
- Output: `gestures` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `hri_msgs/Softbiometrics`
--- ---
### facedetection ### objectdetectionidentificationlocalisation
Node *Face detection* (id: `facedetection`) is overseen by UNITN. Node *Object detection/identification/localisation* (id: `objectdetectionidentificationlocalisation`) is overseen by CVUT.
...@@ -245,19 +246,19 @@ Node *Face detection* (id: `facedetection`) is overseen by UNITN. ...@@ -245,19 +246,19 @@ Node *Face detection* (id: `facedetection`) is overseen by UNITN.
- Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic) - Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic)
- Output: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic) - Output: `/detected_objects [spring_msgs/DetectedObjectArray]` (topic)
#### Dependencies #### Dependencies
- `spring_msgs/DetectedObjectArray`
- `sensor_msgs/Image` - `sensor_msgs/Image`
- `hri_msgs/RegionOfInterestStamped`
--- ---
### robotgui ### voicespeechmatching
Node *Robot GUI* (id: `robotgui`) is overseen by ERM. Node *Voice speech matching* (id: `voicespeechmatching`) is overseen by BIU.
...@@ -267,74 +268,139 @@ Node *Robot GUI* (id: `robotgui`) is overseen by ERM. ...@@ -267,74 +268,139 @@ Node *Robot GUI* (id: `robotgui`) is overseen by ERM.
#### Inputs/outputs #### Inputs/outputs
- Input: `additional support material` (undefined) - Input: `/audio/speech_streams [std_msgs/String]` (topic)
- Input: `/tts/feedback` (undefined) - Input: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
- Input: `speech output` (undefined)
- Input: `/h/v/*/speech [std_msgs/String]` (topic)
- Output: `/h/v/*/speech [hri_msgs/LiveSpeech]` (topic)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `hri_msgs/LiveSpeech`
- `std_msgs/String` - `std_msgs/String`
- `audio_common_msgs/AudioData`
--- ---
### respeaker_ros ### speakeridentification
Node *respeaker_ros* (id: `respeaker_ros`) is overseen by PAL. Node *Speaker identification* (id: `speakeridentification`) is overseen by BIU.
REPO:git@gitlab.inria.fr:spring/wp7_ari/respeaker_ros.git - online services
BIN:respeaker_multichan_node.py - not started yet
#### Status #### Status
**Current release: master** **This node is currently auto-generated (mock-up)**
#### Inputs/outputs #### Inputs/outputs
- Input: `tracking information` (undefined)
- Input: `/audio/postprocess_audio_streams [audio_common_msgs/AudioData]` (topic)
- Output: `/audio/raw_audio [respeaker_ros/RawAudioData]` (topic) - Output: `person_id_candidate [hri_msgs/IdsMatch]` (undefined)
- Output: `/audio/ego_audio [audio_common_msgs/AudioData]` (topic) - Output: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
#### Dependencies #### Dependencies
- `respeaker_ros/RawAudioData` - `std_msgs/Empty`
- `audio_common_msgs/AudioData` - `audio_common_msgs/AudioData`
--- ---
### personreidentification ### mask_detector
Node *Person re-identification* (id: `personreidentification`) is overseen by UNITN. Node *mask_detector* (id: `mask_detector`) is overseen by UNITN.
Detects presence of a facial mask
REPO:git@gitlab.inria.fr:spring/wp4_behavior/wp4_behavior_understanding.git
SUBFOLDER:wp4_people_characteristics
BIN:mask_detector.py
#### Status
**Current release: master**
#### Inputs/outputs
- Input: `/h/f/*/roi [sensor_msgs/RegionOfInterest]` (topic)
- Output: `/h/f/*/mask [std_msgs/Bool]` (topic)
#### Dependencies
- `sensor_msgs/RegionOfInterest`
- `std_msgs/Bool`
---
### hri_person_manager
Node *hri_person_manager* (id: `hri_person_manager`) is overseen by PAL.
REPO:git@gitlab.inria.fr:spring/wp7_ari/hri_person_manager.git
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: master**
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic) - Input: `TF (faces)` (undefined)
- Input: `/h/f/*/softbiometrics [hri_msgs/Softbiometrics]` (topic)
- Input: `candidate_matches [hri_msgs/IdsMatch]` (undefined)
- Input: `TF (voices)` (undefined)
- Output: `person_id_candidate [hri_msgs/IdsMatch]` (undefined) - Output: `/h/p/*/voice_id [std_msgs/String]` (topic)
- Output: `/h/p/*/softbiometrics [hri_msgs/Softbiometrics]` (topic)
- Output: `/h/p/*/face_id [std_msgs/String]` (topic)
- Output: `/humans/persons/*/body_id [std_msgs/String]` (topic)
- Output: `tf: /person_id` (tf)
#### Dependencies #### Dependencies
- `std_msgs/String`
- `std_msgs/Empty` - `std_msgs/Empty`
- `hri_msgs/RegionOfInterestStamped` - `hri_msgs/Softbiometrics`
- `tf/transform_broadcaster`
--- ---
### orbslam ### soft_biometrics_estimator
Node *ORB SLAM* (id: `orbslam`) is overseen by PAL. Node *soft_biometrics_estimator* (id: `soft_biometrics_estimator`) is overseen by UNITN.
Detects age/gender
REPO:git@gitlab.inria.fr:spring/wp4_behavior/wp4_behavior_understanding.git
SUBFOLDER:wp4_people_characteristics
BIN:soft_biometrics_estimator.py
#### Status
**Current release: master**
#### Inputs/outputs
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterest]` (topic)
- Output: `/h/f/*/softbiometrics [hri_msgs/Softbiometrics]` (topic)
#### Dependencies
- `hri_msgs/RegionOfInterest`
- `hri_msgs/Softbiometrics`
---
### rosmediapipeopenpose
Node *ROS mediapipe/openpose* (id: `rosmediapipeopenpose`) is overseen by INRIA.
MOCK: ROS mediapipe/openpose is...
#### Status #### Status
...@@ -342,16 +408,13 @@ Node *ORB SLAM* (id: `orbslam`) is overseen by PAL. ...@@ -342,16 +408,13 @@ Node *ORB SLAM* (id: `orbslam`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `/camera_torso/color/image_raw [sensor_msgs/Image]` (topic) - Input: `input` (undefined)
- Output: `tf: /odom` (tf) - Output: `output` (undefined)
- Output: `/map [nav_msgs/OccupancyGrid]` (topic)
#### Dependencies #### Dependencies
- `tf/transform_broadcaster` - `std_msgs/Empty`
- `sensor_msgs/Image`
- `nav_msgs/OccupancyGrid`
--- ---
...@@ -360,7 +423,7 @@ Node *ORB SLAM* (id: `orbslam`) is overseen by PAL. ...@@ -360,7 +423,7 @@ Node *ORB SLAM* (id: `orbslam`) is overseen by PAL.
Node *People 3D tracker* (id: `people3dtracker`) is overseen by INRIA. Node *People 3D tracker* (id: `people3dtracker`) is overseen by INRIA.
MOCK: People 3D tracker [position/orientation] is...
#### Status #### Status
...@@ -368,9 +431,9 @@ Node *People 3D tracker* (id: `people3dtracker`) is overseen by INRIA. ...@@ -368,9 +431,9 @@ Node *People 3D tracker* (id: `people3dtracker`) is overseen by INRIA.
#### Inputs/outputs #### Inputs/outputs
- Input: `feet position` (undefined)
- Input: `people RoIs` (undefined)
- Input: `ground plane` (undefined) - Input: `ground plane` (undefined)
- Input: `people RoIs` (undefined)
- Input: `feet position` (undefined)
- Output: `tf: /body_id` (tf) - Output: `tf: /body_id` (tf)
...@@ -382,9 +445,9 @@ Node *People 3D tracker* (id: `people3dtracker`) is overseen by INRIA. ...@@ -382,9 +445,9 @@ Node *People 3D tracker* (id: `people3dtracker`) is overseen by INRIA.
--- ---
### fisheye ### fformation
Node *Fisheye* (id: `fisheye`) is overseen by PAL. Node *F-formation* (id: `fformation`) is overseen by UNITN.
...@@ -394,45 +457,47 @@ Node *Fisheye* (id: `fisheye`) is overseen by PAL. ...@@ -394,45 +457,47 @@ Node *Fisheye* (id: `fisheye`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/i/gaze [hri_msgs/Gaze]` (topic)
- Input: `tf: /person_id` (tf)
- Output: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic) - Output: `/h/i/groups [hri_msgs/Group]` (topic)
#### Dependencies #### Dependencies
- `sensor_msgs/Image` - `hri_msgs/Group`
- `hri_msgs/Gaze`
- `tf/transform_listener`
--- ---
### fformation ### respeaker_ros
Node *F-formation* (id: `fformation`) is overseen by UNITN.
Node *respeaker_ros* (id: `respeaker_ros`) is overseen by PAL.
REPO:git@gitlab.inria.fr:spring/wp7_ari/respeaker_ros.git BIN:respeaker_multichan_node.py
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: master**
#### Inputs/outputs #### Inputs/outputs
- Input: `tf: /person_id` (tf)
- Input: `/h/i/gaze [hri_msgs/GazesStamped]` (topic)
- Output: `/h/i/groups [hri_msgs/GroupsStamped]` (topic) - Output: `/audio/ego_audio [audio_common_msgs/AudioData]` (topic)
- Output: `/audio/raw_audio [respeaker_ros/RawAudioData]` (topic)
#### Dependencies #### Dependencies
- `tf/transform_listener` - `audio_common_msgs/AudioData`
- `hri_msgs/GazesStamped` - `respeaker_ros/RawAudioData`
- `hri_msgs/GroupsStamped`
--- ---
### robotfunctionallayer ### fisheye
Node *Robot functional layer* (id: `robotfunctionallayer`) is overseen by PAL. Node *fisheye* (id: `fisheye`) is overseen by PAL.
...@@ -442,23 +507,48 @@ Node *Robot functional layer* (id: `robotfunctionallayer`) is overseen by PAL. ...@@ -442,23 +507,48 @@ Node *Robot functional layer* (id: `robotfunctionallayer`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `input` (undefined)
- Output: `/torso_front_camera/color/image_raw [sensor_msgs/Image]` (topic)
#### Dependencies
- `sensor_msgs/Image`
---
### dialoguearbiter
Node *dialogue arbiter* (id: `dialoguearbiter`) is overseen by HWU.
REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/dialogue.git
SUBFOLDER:dialogue_arbiter
#### Status
**Current release: spring_dev**
#### Inputs/outputs
- Input: `interaction messages` (undefined)
- Input: `/h/v/*/speech [hri_msgs/LiveSpeech]` (topic)
- Output: `next utterance` (undefined)
- Output: `DialogueState` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `hri_msgs/LiveSpeech`
--- ---
### robot_behavior ### facedetection
Node *robot_behavior* (id: `robot_behavior`) is overseen by INRIA. Node *Face detection* (id: `facedetection`) is overseen by UNITN.
REPO: https://gitlab.inria.fr/spring/wp6_robot_behavior/robot_behavior/-/tree/devel/src/robot_behavior
The code is primarily developed at INRIA by Timothée Wintz.
#### Status #### Status
...@@ -466,26 +556,23 @@ The code is primarily developed at INRIA by Timothée Wintz. ...@@ -466,26 +556,23 @@ The code is primarily developed at INRIA by Timothée Wintz.
#### Inputs/outputs #### Inputs/outputs
- Input: `look at` (undefined) - Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic)
- Input: `following/nav goals` (undefined)
- Input: `occupancy map` (undefined)
- Input: `status` (undefined)
- Input: `/h/i/groups [hri_msgs/GroupsStamped]` (topic)
- Output: `status` (undefined) - Output: `/h/f/*/cropped [sensor_msg/Image]` (topic)
- Output: `low-level actions` (undefined) - Output: `/h/f/*/roi [sensor_msgs/RegionOfInterest]` (topic)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `sensor_msg/Image`
- `hri_msgs/GroupsStamped` - `sensor_msgs/RegionOfInterest`
- `sensor_msgs/Image`
--- ---
### userattentionestimation ### depthestimationfrommonocular
Node *User attention estimation* (id: `userattentionestimation`) is overseen by UNITN. Node *Depth estimation from monocular* (id: `depthestimationfrommonocular`) is overseen by UNITN.
...@@ -495,24 +582,20 @@ Node *User attention estimation* (id: `userattentionestimation`) is overseen by ...@@ -495,24 +582,20 @@ Node *User attention estimation* (id: `userattentionestimation`) is overseen by
#### Inputs/outputs #### Inputs/outputs
- Input: `TF (faces)` (undefined) - Input: `input` (undefined)
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic)
- Output: `tf: /face_id_gaze` (tf) - Output: `depth` (undefined)
- Output: `x,y + attention heatmap` (undefined)
#### Dependencies #### Dependencies
- `tf/transform_broadcaster`
- `std_msgs/Empty` - `std_msgs/Empty`
- `hri_msgs/RegionOfInterestStamped`
--- ---
### rgbdcamera_info ### robotfunctionallayer
Node *RGB-D + camera_info* (id: `rgbdcamera_info`) is overseen by PAL. Node *Robot functional layer* (id: `robotfunctionallayer`) is overseen by PAL.
...@@ -522,72 +605,78 @@ Node *RGB-D + camera_info* (id: `rgbdcamera_info`) is overseen by PAL. ...@@ -522,72 +605,78 @@ Node *RGB-D + camera_info* (id: `rgbdcamera_info`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `input` (undefined)
- Output: `/camera_torso/color/image_raw [sensor_msgs/Image]` (topic)
#### Dependencies #### Dependencies
- `sensor_msgs/Image` - `std_msgs/Empty`
--- ---
### nonverbalbehaviours ### fairmotmultipeoplebodytracker
Node *Non-verbal behaviours* (id: `nonverbalbehaviours`) is overseen by UNITN.
Node *FairMOT Multi-people body tracker* (id: `fairmotmultipeoplebodytracker`) is overseen by INRIA.
This code is primarily developed at INRIA by Luis Gomez Camara.
REPO: https://gitlab.inria.fr/spring/wp3_av_perception/multi-person_visual_tracker/
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: devel**
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic) - Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic)
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic)
- Output: `/h/f/*/expression [hri_msgs/Expression]` (topic) - Output: `/h/b/*/cropped [sensor_msg/Image]` (topic)
- Output: `/h/b/*/roi [sensor_msgs/RegionOfInterest` (undefined)
#### Dependencies #### Dependencies
- `hri_msgs/Expression` - `sensor_msg/Image`
- `audio_common_msgs/AudioData` - `sensor_msgs/Image`
- `hri_msgs/RegionOfInterestStamped` - `std_msgs/Empty`
--- ---
### speakeridentification ### mode
Node *Speaker identification* (id: `speakeridentification`) is overseen by BIU. Node *MoDE* (id: `mode`) is overseen by BIU.
- online services This node does:
- not started yet - speech echo cancelation,
- speech enhancement,
- speech separation and diarization
REPO:https://gitlab.inria.fr/spring/wp3_av_perception/speech-enhancement
SUBFOLDER:audio_processing
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: BIU_dev**
#### Inputs/outputs #### Inputs/outputs
- Input: `/audio/postprocess_audio_streams [audio_common_msgs/AudioData]` (topic) - Input: `sound localization` (undefined)
- Input: `tracking information` (undefined) - Input: `/audio/ego_audio [audio_common_msgs/AudioData]` (topic)
- Input: `/audio/raw_audio [respeaker_ros/RawAudioData]` (topic)
- Output: `person_id_candidate [hri_msgs/IdsMatch]` (undefined) - Output: `/audio/postprocess_audio_streams [audio_common_msgs/AudioData]` (topic)
- Output: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
#### Dependencies #### Dependencies
- `audio_common_msgs/AudioData` - `audio_common_msgs/AudioData`
- `std_msgs/Empty` - `std_msgs/Empty`
- `respeaker_ros/RawAudioData`
--- ---
### maskdetection ### speechsynthesis
Node *Mask detection* (id: `maskdetection`) is overseen by UNITN. Node *Speech synthesis* (id: `speechsynthesis`) is overseen by PAL.
...@@ -597,21 +686,20 @@ Node *Mask detection* (id: `maskdetection`) is overseen by UNITN. ...@@ -597,21 +686,20 @@ Node *Mask detection* (id: `maskdetection`) is overseen by UNITN.
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic) - Input: `speech [std_msgs/String]` (undefined)
- Output: `/h/f/*/mask [std_msgs/Bool]` (topic) - Output: `/tts/feedback` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Bool` - `std_msgs/Empty`
- `hri_msgs/RegionOfInterestStamped`
--- ---
### semanticmapping ### uservisualfocus
Node *Semantic mapping* (id: `semanticmapping`) is overseen by CVUT. Node *User visual focus* (id: `uservisualfocus`) is overseen by UNITN.
...@@ -621,29 +709,27 @@ Node *Semantic mapping* (id: `semanticmapping`) is overseen by CVUT. ...@@ -621,29 +709,27 @@ Node *Semantic mapping* (id: `semanticmapping`) is overseen by CVUT.
#### Inputs/outputs #### Inputs/outputs
- Input: `dense 3d map` (undefined) - Input: `scene` (undefined)
- Input: `/detected_objects [spring_msgs/DetectedObjectArray]` (topic) - Input: `attention` (undefined)
- Input: `depth` (undefined)
- Input: `gaze direction` (undefined)
- Output: `scene graph` (undefined) - Output: `who's looking at what?` (undefined)
- Output: `/h/i/gaze [hri_msgs/Gaze]` (topic)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `spring_msgs/DetectedObjectArray` - `hri_msgs/Gaze`
--- ---
### speechanalysis ### orbslam
Node *Speech analysis* (id: `speechanalysis`) is overseen by BIU. Node *ORB SLAM* (id: `orbslam`) is overseen by PAL.
This node does:
- speech echo cancelation,
- speech enhancement,
- speech separation and diarization
REPO: https://gitlab.inria.fr/spring/wp3_av_perception/biu-ros_audio_retreiver
#### Status #### Status
...@@ -651,48 +737,54 @@ REPO: https://gitlab.inria.fr/spring/wp3_av_perception/biu-ros_audio_retreiver ...@@ -651,48 +737,54 @@ REPO: https://gitlab.inria.fr/spring/wp3_av_perception/biu-ros_audio_retreiver
#### Inputs/outputs #### Inputs/outputs
- Input: `sound localization` (undefined) - Input: `/camera_torso/color/image_raw [sensor_msgs/Image]` (topic)
- Input: `/audio/raw_audio [respeaker_ros/RawAudioData]` (topic)
- Input: `/audio/ego_audio [audio_common_msgs/AudioData]` (topic)
- Output: `/audio/postprocess_audio_streams -- cf output in node desc` (undefined) - Output: `/map [nav_msgs/OccupancyGrid]` (topic)
- Output: `tf: /odom` (tf)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `nav_msgs/OccupancyGrid`
- `respeaker_ros/RawAudioData` - `tf/transform_broadcaster`
- `audio_common_msgs/AudioData` - `sensor_msgs/Image`
--- ---
### softbiometrics ### robot_behavior
Node *Soft biometrics* (id: `softbiometrics`) is overseen by UNITN.
Node *robot_behavior* (id: `robot_behavior`) is overseen by INRIA.
The code is primarily developed at INRIA by Timothée Wintz.
REPO: https://gitlab.inria.fr/spring/wp6_robot_behavior/robot_behavior
SUBFOLDER:src/robot_behavior
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: devel**
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/f/*/roi [hri_msgs/RegionOfInterestStamped]` (topic) - Input: `following/nav goals` (undefined)
- Input: `status` (undefined)
- Input: `/h/i/groups [hri_msgs/Group]` (topic)
- Input: `look at` (undefined)
- Input: `occupancy map` (undefined)
- Output: `/h/f/*/demographics [hri_msgs/AgeAndGender]` (topic) - Output: `status` (undefined)
- Output: `low-level actions` (undefined)
#### Dependencies #### Dependencies
- `hri_msgs/AgeAndGender` - `std_msgs/Empty`
- `hri_msgs/RegionOfInterestStamped` - `hri_msgs/Group`
--- ---
### occupancymap ### semanticmapping
Node *Occupancy map* (id: `occupancymap`) is overseen by CVUT. Node *Semantic mapping* (id: `semanticmapping`) is overseen by CVUT.
...@@ -702,50 +794,48 @@ Node *Occupancy map* (id: `occupancymap`) is overseen by CVUT. ...@@ -702,50 +794,48 @@ Node *Occupancy map* (id: `occupancymap`) is overseen by CVUT.
#### Inputs/outputs #### Inputs/outputs
- Input: `TF (bodies)` (undefined)
- Input: `dense 3d map` (undefined) - Input: `dense 3d map` (undefined)
- Input: `/detected_objects [spring_msgs/DetectedObjectArray]` (topic)
- Output: `/map_refined [nav_msgs/OccupancyGrid]` (topic) - Output: `scene graph` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `nav_msgs/OccupancyGrid` - `spring_msgs/DetectedObjectArray`
--- ---
### dialoguearbiter ### social_scene_context_understanding
Node *dialogue arbiter* (id: `dialoguearbiter`) is overseen by HWU. Node *social_scene_context_understanding* (id: `social_scene_context_understanding`) is overseen by HWU.
REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/dialogue/-/tree/spring-dev/dialogue_arbiter REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/interaction.git
SUBFOLDER:social_scene_context_understanding
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: spring_dev**
#### Inputs/outputs #### Inputs/outputs
- Input: `interaction messages` (undefined) - Input: `scene graph` (undefined)
- Input: `/h/v/*/speech [std_msgs/String]` (topic)
- Output: `next utterance` (undefined) - Output: `semantic description` (undefined)
- Output: `DialogueState` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `std_msgs/String`
--- ---
### plan_actions ### robotgui
Node *Robot GUI* (id: `robotgui`) is overseen by ERM.
Node *plan_actions* (id: `plan_actions`) is overseen by HWU.
REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/plan_actions
#### Status #### Status
...@@ -753,53 +843,48 @@ REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/plan_actions ...@@ -753,53 +843,48 @@ REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/plan_actions
#### Inputs/outputs #### Inputs/outputs
- Input: `TF (persons)` (undefined) - Input: `speech output` (undefined)
- Input: `semantic scene description` (undefined) - Input: `additional support material` (undefined)
- Input: `demographics` (undefined) - Input: `/h/v/*/speech [hri_msgs/LiveSpeech]` (topic)
- Input: `dialogue state` (undefined) - Input: `/tts/feedback` (undefined)
- Output: `interaction state` (undefined)
- Output: `output` (undefined)
- Output: `nav goals` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `hri_msgs/LiveSpeech`
--- ---
### uservisualfocus ### ros_openpose
Node *User visual focus* (id: `uservisualfocus`) is overseen by UNITN.
Node *ros_openpose* (id: `ros_openpose`) is overseen by UNITN.
REPO:git@gitlab.inria.fr:spring/wp4_behavior/wp4_behavior_understanding.git
SUBFOLDER:ros_openpose
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: 0.0.1**
#### Inputs/outputs #### Inputs/outputs
- Input: `attention` (undefined) - Input: `/h/b/*/cropped [sensor_msg/Image]` (topic)
- Input: `scene` (undefined)
- Input: `gaze direction` (undefined)
- Input: `depth` (undefined)
- Output: `who's looking at what?` (undefined) - Output: `/h/b/*/skeleton2d [hri_msgs/Skeleton2D]` (topic)
- Output: `/h/i/gaze [hri_msgs/GazesStamped]` (topic)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `sensor_msg/Image`
- `hri_msgs/GazesStamped` - `hri_msgs/Skeleton2D`
--- ---
### speechsynthesis ### activityreco
Node *Speech synthesis* (id: `speechsynthesis`) is overseen by PAL. Node *Activity reco* (id: `activityreco`) is overseen by UNITN.
...@@ -809,9 +894,10 @@ Node *Speech synthesis* (id: `speechsynthesis`) is overseen by PAL. ...@@ -809,9 +894,10 @@ Node *Speech synthesis* (id: `speechsynthesis`) is overseen by PAL.
#### Inputs/outputs #### Inputs/outputs
- Input: `speech [std_msgs/String]` (undefined) - Input: `TF (bodies)` (undefined)
- Input: `gaze direction` (undefined)
- Output: `/tts/feedback` (undefined) - Output: `[?] output` (undefined)
#### Dependencies #### Dependencies
...@@ -820,32 +906,39 @@ Node *Speech synthesis* (id: `speechsynthesis`) is overseen by PAL. ...@@ -820,32 +906,39 @@ Node *Speech synthesis* (id: `speechsynthesis`) is overseen by PAL.
--- ---
### sceneunderstanding ### robot_behaviour_plan_actions
Node *Scene understanding* (id: `sceneunderstanding`) is overseen by HWU.
Node *robot_behaviour_plan_actions* (id: `robot_behaviour_plan_actions`) is overseen by HWU.
REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/plan_actions.git
SUBFOLDER:robot_behaviour_plan_actions
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: spring_dev**
#### Inputs/outputs #### Inputs/outputs
- Input: `scene graph` (undefined) - Input: `TF (persons)` (undefined)
- Input: `semantic scene description` (undefined)
- Input: `dialogue state` (undefined)
- Input: `/h/p/*/softbiometrics [hri_msgs/Softbiometrics]` (topic)
- Output: `semantic description` (undefined) - Output: `nav goals` (undefined)
- Output: `interaction state` (undefined)
- Output: `output` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `std_msgs/Empty`
- `hri_msgs/Softbiometrics`
--- ---
### rosopenpose ### multipersonaudiotracking
Node *ROS openpose* (id: `rosopenpose`) is overseen by INRIA. Node *Multi-person audio tracking* (id: `multipersonaudiotracking`) is overseen by BIU.
...@@ -855,20 +948,25 @@ Node *ROS openpose* (id: `rosopenpose`) is overseen by INRIA. ...@@ -855,20 +948,25 @@ Node *ROS openpose* (id: `rosopenpose`) is overseen by INRIA.
#### Inputs/outputs #### Inputs/outputs
- Input: `input` (undefined) - Input: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
- Output: `tracking information` (undefined)
- Output: `output` (undefined) - Output: `output` (undefined)
- Output: `source angle` (undefined)
- Output: `tf: /voice_id` (tf)
#### Dependencies #### Dependencies
- `audio_common_msgs/AudioData`
- `std_msgs/Empty` - `std_msgs/Empty`
- `tf/transform_broadcaster`
--- ---
### visualslam3dmap ### inlocros
Node *Visual SLAM + 3D map* (id: `visualslam3dmap`) is overseen by CVUT. Node *InLoc-ROS* (id: `inlocros`) is overseen by CVUT.
...@@ -878,8 +976,8 @@ Node *Visual SLAM + 3D map* (id: `visualslam3dmap`) is overseen by CVUT. ...@@ -878,8 +976,8 @@ Node *Visual SLAM + 3D map* (id: `visualslam3dmap`) is overseen by CVUT.
#### Inputs/outputs #### Inputs/outputs
- Input: `/camera_head/color/image_head [sensor_msgs/Image]` (topic)
- Input: `/camera_torso/color/image_torso [sensor_msgs/Image]` (topic) - Input: `/camera_torso/color/image_torso [sensor_msgs/Image]` (topic)
- Input: `/camera_head/color/image_head [sensor_msgs/Image]` (topic)
- Input: `localisation prior` (undefined) - Input: `localisation prior` (undefined)
- Output: `dense 3d map` (undefined) - Output: `dense 3d map` (undefined)
...@@ -887,42 +985,43 @@ Node *Visual SLAM + 3D map* (id: `visualslam3dmap`) is overseen by CVUT. ...@@ -887,42 +985,43 @@ Node *Visual SLAM + 3D map* (id: `visualslam3dmap`) is overseen by CVUT.
#### Dependencies #### Dependencies
- `sensor_msgs/Image`
- `std_msgs/Empty` - `std_msgs/Empty`
- `tf/transform_broadcaster` - `tf/transform_broadcaster`
- `sensor_msgs/Image`
--- ---
### objectdetectionidentificationlocalisation ### google_asr
Node *Object detection/identification/localisation* (id: `objectdetectionidentificationlocalisation`) is overseen by CVUT.
Node *google_asr* (id: `google_asr`) is overseen by BIU.
REPO:https://gitlab.inria.fr/spring/wp5_spoken_conversations/asr
SUBFOLDER:google_asr/google_asr
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: BIU_dev**
#### Inputs/outputs #### Inputs/outputs
- Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic) - Input: `/audio/postprocess_audio_streams [audio_common_msgs/AudioData]` (topic)
- Output: `/detected_objects [spring_msgs/DetectedObjectArray]` (topic) - Output: `/audio/speech_streams -- array of hri_msgs/LiveSpeech` (undefined)
#### Dependencies #### Dependencies
- `sensor_msgs/Image` - `audio_common_msgs/AudioData`
- `spring_msgs/DetectedObjectArray` - `std_msgs/Empty`
--- ---
### body2dpose ### occupancymap
Node *Occupancy map* (id: `occupancymap`) is overseen by CVUT.
Node *Body 2D pose* (id: `body2dpose`) is overseen by UNITN.
MOCK: Body 2D pose [openpose] is...
#### Status #### Status
...@@ -930,25 +1029,24 @@ MOCK: Body 2D pose [openpose] is... ...@@ -930,25 +1029,24 @@ MOCK: Body 2D pose [openpose] is...
#### Inputs/outputs #### Inputs/outputs
- Input: `/h/b/*/roi [hri_msgs/RegionOfInterestStamped]` (topic) - Input: `dense 3d map` (undefined)
- Input: `/camera_head/color/image_raw [sensor_msgs/Image]` (topic) - Input: `TF (bodies)` (undefined)
- Output: `/h/b/*/skeleton2d [hri_msgs/Skeleton2D]` (topic) - Output: `/map_refined [nav_msgs/OccupancyGrid]` (topic)
#### Dependencies #### Dependencies
- `hri_msgs/RegionOfInterestStamped` - `std_msgs/Empty`
- `sensor_msgs/Image` - `nav_msgs/OccupancyGrid`
- `hri_msgs/Skeleton2D`
--- ---
### interaction_manager ### raspicam
Node *raspicam* (id: `raspicam`) is overseen by PAL.
Node *interaction_manager* (id: `interaction_manager`) is overseen by HWU.
REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/interaction/-/tree/spring_dev/interaction_manager
#### Status #### Status
...@@ -956,94 +1054,102 @@ REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/interaction/-/tree ...@@ -956,94 +1054,102 @@ REPO: https://gitlab.inria.fr/spring/wp5_spoken_conversations/interaction/-/tree
#### Inputs/outputs #### Inputs/outputs
- Input: `semantic scene description` (undefined)
- Input: `input` (undefined)
- Input: `dialogue state` (undefined)
- Input: `robot state` (undefined)
- Input: `TF` (undefined)
- Input: `/h/p/personID` (undefined)
- Output: `who to look at` (undefined) - Output: `/head_front_camera/color/image_raw [sensor_msgs/Image]` (topic)
- Output: `verbal command` (undefined)
- Output: `nav goals` (undefined)
- Output: `active personID` (undefined)
- Output: `gestures` (undefined)
#### Dependencies #### Dependencies
- `std_msgs/Empty` - `sensor_msgs/Image`
---
### personmanager ### Non-executable dependency: interaction_manager_msgs
Module interaction_manager_msgs (id: `interaction_manager_msgs`) is overseen by HWU.
Node *Person manager* (id: `personmanager`) is overseen by INRIA. REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/interaction.git
SUBFOLDER:interaction_manager_msgs
NOT EXECUTABLE
PAL to write an initial temporary version, simply forwarding detected faces as persons.
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: spring_dev**
#### Inputs/outputs #### Dependencies
- Input: `TF (voices)` (undefined)
- Input: `TF (faces)` (undefined)
- Input: `candidate_matchs [hri_msgs/IdsMatch]` (undefined)
- Input: `/h/f/*/demographics [hri_msgs/AgeAndGender]` (topic)
- Output: `tf: /person_id` (tf)
- Output: `/humans/persons/*/body_id [std_msgs/String]` (topic)
- Output: `/h/p/*/voice_id [std_msgs/String]` (topic)
- Output: `/h/p/*/demographics [hri_msgs/AgeAndGender]` (topic) ### Non-executable dependency: hri_msgs
- Output: `/h/p/*/face_id [std_msgs/String]` (topic)
Module hri_msgs (id: `hri_msgs`) is overseen by PAL.
REPO: git@gitlab:ros4hri/hri_msgs.git
NOT EXECUTABLE
#### Status
**Current release: 0.1.1**
#### Dependencies #### Dependencies
- `tf/transform_broadcaster`
- `std_msgs/String`
- `std_msgs/Empty`
- `hri_msgs/AgeAndGender`
---
### voicespeechmatching
Node *Voice speech matching* (id: `voicespeechmatching`) is overseen by BIU. ### Non-executable dependency: robot_behaviour_msgs
Module robot_behaviour_msgs (id: `robot_behaviour_msgs`) is overseen by HWU.
REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/plan_actions.git
SUBFOLDER:robot_behaviour_msgs
NOT EXECUTABLE
#### Status #### Status
**This node is currently auto-generated (mock-up)** **Current release: spring_dev**
#### Dependencies
#### Inputs/outputs
- Input: `/audio/speech_streams [std_msgs/String]` (topic)
- Input: `/h/v/*/audio [audio_common_msgs/AudioData]` (topic)
- Output: `/h/v/*/speech [std_msgs/String]` (topic) ### Non-executable dependency: audio_msgs
Module audio_msgs (id: `audio_msgs`) is overseen by HWU.
REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/asr.git
SUBFOLDER:audio_msgs
NOT EXECUTABLE
#### Status
**Current release: spring_dev**
#### Dependencies #### Dependencies
- `std_msgs/String`
- `audio_common_msgs/AudioData`
### Non-executable dependency: spring_msgs
Module spring_msgs (id: `spring_msgs`) is overseen by Other. ### Non-executable dependency: social_scene_msgs
Module social_scene_msgs (id: `social_scene_msgs`) is overseen by HWU.
REPO:git@gitlab.inria.fr:spring/wp7_ari/spring_msgs.git REPO:git@gitlab.inria.fr:spring/wp5_spoken_conversations/interaction.git
SUBFOLDER:social_scene_msgs
NOT EXECUTABLE NOT EXECUTABLE
#### Status #### Status
**Current release: 0.0.2** **Current release: spring_dev**
#### Dependencies #### Dependencies
...@@ -1051,17 +1157,18 @@ NOT EXECUTABLE ...@@ -1051,17 +1157,18 @@ NOT EXECUTABLE
### Non-executable dependency: hri_msgs ### Non-executable dependency: wp4_msgs
Module hri_msgs (id: `hri_msgs`) is overseen by Other. Module wp4_msgs (id: `wp4_msgs`) is overseen by UNITN.
REPO: git@gitlab:ros4hri/hri_msgs.git REPO:git@gitlab.inria.fr:spring/wp4_behavior/wp4_behavior_understanding.git
SUBFOLDER:wp4_msgs
NOT EXECUTABLE NOT EXECUTABLE
#### Status #### Status
**Current release: 0.1.1** **Current release: master**
#### Dependencies #### Dependencies
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment