The ROS2 Monitoring component is meant for developers using ROS2: a dashboard for monitoring health, login, examining services, publishers and subscribers associated to ROS2 nodes. The component includes a GUI, which is used to interact with the component, through the GUI the user can setup the ROS components that the user wants to monitor.
Main non-functional requirements
The component has no real-time responsiveness requirements
Software requirements/dependencies
Platform: Ubuntu, MacOS, Windows Requirements: ROS2, Docker
Hardware requirements
64-bit system capable of running: Ubuntu, MacOS, Windows and ROS2
Security threats
The component should operate behind a firewall during production
Privacy threats
No privacy threats have been identified
Execution place
Private Cloud/PC neart production
Deployment instructions
Deployment instructions can be found on a public repository
User interface
A dashboard showing the current status of all ROS2 nodes in the system
Supported devices
Desktop/Laptop, XX
User defined scenarios (non-technical) and relevant pilot cases
The component can be used to monitor a collection of ROS2 nodes
The main functionality of this component is to enable robot control using natural human actions as input, in this case hand gestures using a gesture tracking glove. With sensor glove by CaptoGlove LLC, the operator makes distinguishable hand gestures to command and control the robot in the process. The component is reconfigurable for different controlling scenarios.
Main non-functional requirements
N/A
Software requirements/dependencies
For the CaptoGlove, there should be Capto Suite installed. Docker installed on the host machine
Hardware requirements
20GB Hard drive space, recommended 2GB RAM
Security threats
None
Privacy threats
None
Execution place
CaptoGlove SDK is ran on host Windows PC and all the other parts of M2O2P are docker images
Deployment instructions
Instructions of application are provided in PDF format, later on video too.
User interface
User interface for the component is mainly in Web UI of the component. This can be reached from host machine navigating to localhost:54400 on browser.
Supported devices
Any Windows 10 machine, CaptoGlove
User defined scenarios (non-technical) and relevant pilot cases
Component can be used in any system where there is a need to send commands or finish tasks by human operator using the glove. In the Siemens pilot case, the component was used to complete tasks in a bin picking collaborative robotic application.
The main functionality of this component is to enable the training and support of human workers in collaborative tasks. For doing so, the main activities of the collaborative task and the interaction of worker and robot is created in Virtual Reality (VR). By using a virtual reality headset and equipment the worker can remotely visualize, monitor, and perform the training of collaborative tasks with robots. It should be noted that based on the use case requirements (e.g., workspace and environment, equipment, safety aspects and interfaces to other components), several data inputs might be needed for creation of custom simulations. The component is divided into a sandbox mode (using pre-programmed actions) and a dynamic mode, which depending on configuration could receive data inputs from ROS nodes for on-the-fly creation of tasks
Main non-functional requirements
N/A
Software requirements/dependencies
Windows 10 and compatible browser (Firefox, Chrome, etc)
Hardware requirements
A VR headset supported by A-Frame with controller positional tracking, as listed here: https://aframe.io/docs/1.2.0/introduction/vr-headsets-and-webvr-browsers.html
Security threats
None
Privacy threats
None
Execution place
Private cloud (meaning in pilot premises), cloud
Deployment instructions
Provide information on where deployment instructions for a ready component can be found (e.g. on public or private access repositories or on websites or only upon request, etc.)
User interface
A main configuration page and a sample workcell layout in VR mode.
Supported devices
Desktops, Laptops
User defined scenarios (non-technical) and relevant pilot cases
This component could be used to train workers in a collaborative assembly process by virtualizing the whole procedure in VR and allowing the worker to interact with the robot and components prior to working in the actual setup. It is important to keep in mind that since this application is meant for training, having a concrete step by step process is required to design and fully benefit the collaborative training
The Digital Twin for control and planning (DT-CP) allows the users to create a virtual replica of the production line facilitated by the use case. The component is divided into two parts: A simulator, to experiment with alternative models to be implemented in the real line, and a monitoring dashboard for having an overview of the line.
The time-based simulator takes as input several configuration parameters (e.g., takt time, shift time), process descriptions and resources (e.g., workers) with different skill set. Hence the user can modify and test different production strategies that would be more complicated and time consuming to test in the real line. The monitoring dashboard can provide the user with an overview on the status of production and provide notification mechanism for alerts when implemented. It should be noted that some functionalities (e.g., data sources for monitoring dashboard, data modelling, and configuration parameters for simulator) are dependent on the use case and might require further adaptations for proper integration in the setup.
Main non-functional requirements
N/A
Software requirements/dependencies
N/A
Hardware requirements
Device capable of handling web-based applications
Security threats
None
Privacy threats
None
Execution place
Private cloud (meaning in pilot premises), cloud
Deployment instructions
Instructions will be provided on the Git page
User interface
The component will be divided into two parts: an online dashboard for monitoring the line in real time and a simulator, to experiment with alternative models to be implemented in the real line.
The monitoring dashboard is intended to follow the flow of product from one workstation to another.
The simulator allows to modify and test different production strategies that would be more complicated and time consuming to test in the real line. The simulator setup consists of four steps: the introduction of the initial process execution parameters, design of the layout, process description assignment and allocation of resources.
Supported devices
Desktop, Laptop
User defined scenarios (non-technical) and relevant pilot cases
Coupled with data collection applications, the monitoring part of the component could be used to have an overview on the process and provide notification and control mechanisms. Data inputs, notification handling, and control mechanisms may be to be further adapted as per use case.
The simulator allows the user to define a time-based simulation of a process assembly by setting concrete configuration parameters (e.g., takt time, shift time) and assigning resources (e.g., workers) with additional resource parameters (e.g., skill set). Th result would be a report on the pre-provided production targets. Specific configuration parameters and settings may have to be further adapted as per use case.
The Data Collection Framework (DCF) component collects data from shop floor (field devices, sensors, and controllers) and enterprise resource planning (ERP) systems using data adapters for different use cases. DCF can be used at production/assembly lines to have an overview on the collected data from sensors and workstations. The main function of the component is data collection from systems, data storage into databases if needed, and for engineers/supervisor to review and take appropriate action. It should be noted that some of the data adapters (e.g., ERP adapters) may need to be configured and tested on a use case basis.
Main non-functional requirements
N/A
Software requirements/dependencies
Data Transfer and Communication within DCF and Database requires Python installed and some Libraries (opcua, paho.mqtt, pymongo, pandas, json, flask, requests, cx_Oracle, hbdcli)
Hardware requirements
– Windows 7 or 10
– x86 64-bit CPU (Intel / AMD architecture)
– 4 GB RAM
-5 GB free disk space
Security threats
The component requires authentication from server/database before connecting and collecting ERP or shopfloor data and storing the data in database.
Privacy threats
None
Execution place
Both devices connected with local network and on different host address can be connected via MQTT and OPC-UA
Deployment instructions
DCF component will be deployed in docker and relevant instructions will be provided.
User interface
After specifying necessary connection configuration, the DCF module is monitoring the temperature and pressure reading through opc-ua server (left image). If the temperature or pressure is more than the allowed, it is logging the information (e.g. time, value, description) in the database (right image). These parameters can be changed according to the use case.
Supported devices
Desktop, Laptop
User defined scenarios (non-technical) and relevant pilot cases
DCF component can be used at production/assembly lines to collect data from workstation/sensors and apply event processing. For instance, if more time is being consumed to complete the task at specific workstation, this activity can be monitored, and relevant data can be logged in database for engineers/supervisor to view and take appropriate action
This component creates user interfaces depending on the information collected from the production line devices and the user’s profile. By doing so, relevant task and operation specific user interfaces are composed for the user. Such interfaces could for example display task specific work description (e.g, description of assembly operation) to users and enable the user to confirm completion of task for interaction with external components. It should be noted that if applicable, other interfaces with different functionalities may have to be developed based on the requirements
Main non-functional requirements
N/A
Software requirements/dependencies
N/A
Hardware requirements
Device capable of use web-based application (e.g.: PC or laptop )
Security threats
None
Privacy threats
None
Execution place
Private cloud (meaning in pilot premises)
Deployment instructions
Instructions will be provided on the Git page
User interface
Supported devices
Desktop, Laptop
User defined scenarios (non-technical) and relevant pilot cases
ADIN can be used by workers in an assembly line for assisting them on the task, giving them the specific and relevant information for fulfilling the duty.
It also can be used in collaborative task with cobots where the worker receive instructions on the steps of the collaboration task.
Human-Centered Process Optimization based on RL in which its main functionality is to provide Reinforcement Learning logic wrapped in ROS2 packages
Main non-functional requirements
Requirements for real-time responsiveness depends on the application. However, since reinforcement learning optimize next action and not the current, real-time responsiveness requirements are relaxed.
Software requirements/dependencies
Platform: Ubuntu, macOS, Windows.
Requirements: ROS2, Docker
Hardware requirements
64-bit system capable of running: Ubuntu, macOS, Windows.
High performance PC/Cloud (good CPU and GPU) for training models.
Eg: +10 core count, +64 GB ram, RTX 2080ti for training (depending on the application and model).
Security threats
When deployed on-premise a firewall should be enough. If deployed in the cloud work is required to ensure a secure connection between the cloud and the production equipment/PC.
Privacy threats
No privacy threats have been identified.
Execution place
Private cloud/PC near robot.
Deployment instructions
Deployment instructions for the component can be found on a private access repository.
User interface
The component will have multiple user interfaces:
A common user interface (dashboard) for developers and end-users containing data visualization, selected actions, and other diagnostics.
Developers will furthermore have a GUI for yaml-file system configuration and all available ROS2 tools for visualization and diagnostics.
Supported devices
Desktop/Cloud
User defined scenarios (non-technical) and relevant pilot cases
The component can be used to optimize a work cell process with reinforcement learning. Example: optimize the process control of a vibration feeder, such that that an element always is available for a robot to pick up.
The component is based on a generic add-on force-control for classical industrial and/or collaborative robots.
An innovative force-sensor based strategy is used to fit two or more parts together that require a snap connection.
The component is a ROS based control approach.
Main non-functional requirements
– Low latency required
– The trained assembly skills can be scaled in time and are primarily limited by the force control performance of the robot (F/T sensor)
Software requirements/dependencies
– ROS framework with ROS control (kinetic or melodic)
– FZI Custom extension of ROS Cartesian Motion, Impedance and Force Controllers
– FZI Custom wrappers for external robotic sensors – Robot ROS driver (e.g. ROS UR)
– TensorFlow 2.1 with python 2.7
Hardware requirements
– Robot with wrist force-torque sensor mounted or integrated
– Dedicated Pc (e.g. i7 shuttle PC with 8 GB ram)
Security threats
The component should run on a separate network without access to the public internet or to any other network not authorized to use it (ROS1 security)
Privacy threats
No specific privacy requirements, no personal information logging
Execution place
Local
Deployment instructions
Internal development, deployment instructions only upon (approved) request
User interface
Text configuration files
Supported devices
Any robot which supports ROS control and can measure end-effector forces and torques (intrinsic or integrated)
User defined scenarios (non-technical) and relevant pilot cases
Force-based assembly tasks which require difficult snap fitting of parts by a robot
Pilot cases: Siemens use case 1
Task manager for safe and efficient human-robot interaction
Main non-functional requirements
– Real time responsiveness is fundamental for the task scheduling to work safely and properly. The supervision of the robot pose requires at least 10 checks per second of the environment, for the robot to accurately react if there is any obstacle on its way
– Sensor information (in particular depth information) needs to be as up-to-date as possible
Software requirements/dependencies
– ROS1 Framework
– GPU-Voxels
– FZI Custom extension of ROS Cartesian Motion, Impedance and Force Controllers
– FZI Specific Extension of the FlexBE ROS package or FZI behaviour-Tree Implementation for Task Modelling and Scheduling
– FZI Custom ROS wrappers for external robotic sensors
– FZI Shared workspace (ROS application of GPU-Voxels) for human-robot-collaboration
– FZI Robot Collision Detection ROS package
– FZI Human Pose Prediction and Tracking software (optional)
– Robot ROS Driver
Hardware requirements
– (Depth) Cameras with fast update rate for the images
– Combination of several sensors (one is not enough)
– 1 shuttle PC for robot control with real time optimization (low latency)
– 1 additional PC with GPU for more computational intense tasks (i.e. collision avoidance, human detection)
Security threats
Run on a separate network (ROS1 security) without access to the public internet or to any network not authorized to use it
Privacy threats
No specific privacy requirements. No personal information, camera or 3D data logging
Execution place
Local
Deployment instructions
Internal development, deployment instructions only upon (approved) request
User interface
Text configuration files
Supported devices
Any robot with ROS driver, URDF description and real time joint angles
User defined scenarios (non-technical) and relevant pilot cases
Efficient Human-Robot collaboration on the shop floor, where the robot needs to fulfil tasks in the proximity of the worker
Pilot Use case: Siemens Use Case 1
Main non-functional requirements
– Inputs expected at 10 Hz. Outputs between 2 and 10 Hz
– Lower frequencies will influence safety and acceptability severely
Software requirements/dependencies
– ROS1 framework
– Google Cartographer ROS
– Move_Base and/or Move_Base_Flex ROS packages
– AGV ROS driver
– External ROS sensor drivers (cameras, lasers)
– Open Pose
– Wheel Odometry (ROS Topic)
Hardware requirements
– SICK Lidar (for example, SICK)
– Intel RealSense and/or 2D camera
– Dedicated PC (Intel5, High End GPU)
Security threats
Operates inside of mobile robot or secured WIFI connection
(no off premises connection required)
Privacy threats
No personal information logging
Execution place
Local
Deployment instructions
Present: Internal development available on approved request
Future: Public access repositories
User interface
– Text configuration files
– (optional) GUI
Supported devices
– Specific PC (to be embedded in a compatible AGV)
– Any AGV with ROS driver
User defined scenarios (non-technical) and relevant pilot cases
Mobile Robot evolving in an industrial plant or public area with people
Pilot use case: Bosch use cases 1 and 2
Graphical front end (GUI) to program new robotic applications by quickly creating new control sequences based on ROS tools.
The tool helps to develop or change the collaborative robotic applications, gives monitoring feedback on the status of the process and could be used to model different tasks as well as the interaction between robot and human transparently.
It is an alternative to SMACH and FlexBE using Behavior Trees.
Main non-functional requirements
No real time responsiveness required
Software requirements/dependencies
ROS1 Framework
Hardware requirements
A PC
Security threats
Run on a separate network (ROS1 security) without access to the public internet or to any network not authorized to use it
Privacy threats
No specific privacy requirements, no personal information logging
Execution place
Local
Deployment instructions
Internal development, deployment instructions only upon (approved) request
User interface
HTML editor to control the functionalities of the task-programming tool
Supported devices
GUI: Any device that allows mouse-like controls
User defined scenarios (non-technical) and relevant pilot cases
Any scenario which involves programming of robots
Pilot use cases: Siemens use cases 1 and 2, Bosch use case 1
This component is used to determine whether the chosen robot trajectory & speed is safe and the required separation distance has been chosen adequately and can be covered by the sensor configuration. (This uses a calculation of the size of the required separation distances for robots that use the operating mode speed and separation monitoring.)
Main non-functional requirements
Trajectories should be checked as early as possible to minimize the delay of execution. Ideally, precomputed trajectories are validated in advance.
Software requirements/dependencies
Currently Visual Components together with the IFF Safety Planning Tool are required for setting up the cell layout. In the future, other tools for this process might be available.
To activate all features and use optimized calculations, a valid license can be purchased from Fraunhofer IFF.
The component runs as a Linux Docker Container on Linux and Windows hosts.
Hardware requirements
PC, no special performance features
Security threats
no known issues
Privacy threats
no known issues (no cameras, no collection or processing of personal data)
Execution place
PC next to robot cell or server. Other options possible (e.g. Private cloud).
Deployment instructions
Deployment instructions will be available on the Shop4CF Docker Registry (docker.ramp.eu)
User interface
No user interface available.
The REST API is documented using Swagger.
Supported devices
PC, Docker
User defined scenarios (non-technical) and relevant pilot cases
When operating a robot cell that uses speed and separation monitoring for safety purposes, you have to check if a given trajecory is safe. If the trajectories are fixed or worst case trajectories can be defined, the operator can check them during design phase (e.g. using the IFF Safety Planning Tool). If trajectories can change (e.g. when using dynamic motion planing), the ASA component allows you to check if a trajectory is safe and the separation distance can be monitored by the sensor configuration. If a trajectory is not safe, the user can calculate a different one or reduce the robot’s speed until all safety conditions are met.
The ASA component ist utilized in the Siemens Use Case 1, where collision-free trajectories are calculated at run-time.
The review of the risk analysis supports a safety expert in identifying hazards and estimating risk. The responsible human designer is guided through the formalized process of identifying new hazards based on identified or manually captured system changes (e.g. part changes including geometry and payload; robot changes including speed, reach, tooling; environmental changes including new tables, fencing, etc.). Application highlights where the existing risk estimation requires updates.
Main non-functional requirements
Bidirectional network access from RA server to FIWARE server (if not used as standalone tool)
Port forwarding, firewall configuration
Secret injection via files or environment variables
Software requirements/dependencies
Container runtime, e.g. Docker
Hardware requirements
Server: about 50 MB free RAM / < 1GB disk space.
End-user device: min. 1280×720, ideally 1920×1080 screen, modern web-browser, about 500MB free RAM
Security threats
Production application must use HTTPS reverse proxy. Secrets must be generated and injected securely. Network access should ideally be limited. Additional requirements apply depending on threat model. E.g. when using FIWARE integration and local user management with enabled self-registration, the server must be located inside secure network (on the same permissions level as FIWARE server) due to granted lateral access to FIWARE server (alternatively use authentication via identity server for managing trust).
Privacy threats
Process critical data could be part of the data sent between client and server and this could put partners’ data at risk.
Execution place
Gateway, private cloud (meaning in pilot premises), cloud, etc.
Unrestricted
Deployment instructions
Deployment instructions are a part of the overall documentation and are included in container and available as separate PDF.
User interface
Browser based interface with multiple tabs, available in German and English (with possibility to add additional languages). Documentation (included in container and available as PDF), video see below.
Supported devices
The server host is unrestricted. The end-user device should preferably be laptop or PC (due to size and amount of information on the screen).
User defined scenarios (non-technical) and relevant pilot cases
See ”Main functions”. Additionally, in the FIWARE configuration monitoring mode the RA component will track the resource assignment to the Process on FIWARE server and either automatically communicate previous approval of the configuration or facilitate review by the safety expert.
The aim of the FLINT platform is to facilitate the incorporation of current/future wireless IoT devices (sensors/actuators) in a factory or shopfloor setting, as well as the required local wireless IoT communication infrastructure to connect such devices (e.g. LoRa gateways, BLE gateways). This component requires horizontal integration. At the left side, it will make use of adapters to interact with wireless IoT devices and long-range wireless communication equipment. At the right, after performing the required data transformations, it will either represent the IoT device as a LwM2M compliant device that can interface in a standardized way with a LwM2M back-end platform (for instance the open source Leshan platform) or deliver the data in a suitable format to a broker (e.g. FIWARE context broker).
Main non-functional requirements
N/A
Software requirements/dependencies
– Dependency on data formats used by IoT devices / used wireless IoT infrastructure, which requires the 1-time design of suitable input/processing adapters. Similar dependency for output adapters in case no processing to LwM2M.
– Docker: adapters realized as Docker containers. Implementation of adapters can be done in any language.
– MQTT broker: for the information exchange between adapters
– LwM2M processing adapters: dependency on Anjay, a C client implementation of LwM2M
Hardware requirements
Server/cloud platform supporting deployment/management of Docker containers.
Security threats
Currently, the internal communication between the adapters and MQTT broker is not secured. However most deployments are done on a secure company network, so the security risk should be limited.
Privacy threats
Privacy threats will depend on the type of data that is collected by IoT devices)
Execution place
Private Cloud
Deployment instructions
Deployment instructions can be found on https://github.com/imec-idlab/flint. Customization will be needed depending on the IoT devices/infrastructure to be used/deployed.
User interface
Dashboard for monitoring data received from / sent to IoT devices (see screenshot below). However, the user interface is not the core of the component, as it can operate without any UI.
Supported devices
The aim of the platform is to be extensible to support a wide range of wireless IoT devices and technologies.
User defined scenarios (non-technical) and relevant pilot cases
Industrial monitoring, asset tracking, environmental monitoring, etc.
Supporting human workers on the shop floor by giving them real-time wireless control over aspects such as process management, interactions with robots, collecting sensor data.
Main non-functional requirements
N/A
Software requirements/dependencies
Linux OS, GNU toolchain, Xilinix Toolchain
Hardware requirements
SDR board (e.g. Xilinx ZC706 + FMCOMMS2/3/4 or other compliant board see https://github.com/open-sdr/openwifi )
Security threats
WPA2 encryption is available and should be sufficient. Of course, a network firewall is necessary.
Privacy threats
All data transmitted over the same WiFi network can be seen by all connected clients. So SSL encryption might be necessary.
Execution place
Private cloud (meaning in pilot premises)
Deployment instructions
All information and source code is available on https://github.com/open-sdr/openwifi
User interface
– Developer: interact with openWIFI through linux WiFi driver (e.g. ath9k), and interface to openwifi specific components with a command line program (“sdrctl”)
– User: openWiFi acts as a regular WiFi access point
Supported devices
All 802.11 WiFi enabled devices are supported (smartphones, tablet, laptops, embedded WiFi hardware, WiFi sensors, …)
User defined scenarios (non-technical) and relevant pilot cases
The Wi-POS system is able to accurately determine the position of AGVs, robots or equipment on the SHOP floor. Positioning workers is also possible, but might be difficult for pri