Autonomous Systems & Mobility: How Machines Perceive Their Environment and Act Independently
Autonomous systems can perform tasks without continuous human control. They perceive their surroundings, assess situations, make decisions and translate those decisions into actions.
In mobility, the spectrum ranges from driver-assistance systems and self-driving vehicles to autonomous delivery robots, drones, ships and mobile machinery.
The foundation consists of sensing, artificial intelligence, precise localisation and reliable control. Only the interaction of these components enables a system to move safely through a dynamic environment.
What Is an Autonomous System?
An autonomous system can act independently within defined limits.
To do so, it must perform several tasks:
perceive the environment,
determine its own position,
identify relevant objects and situations,
predict future developments,
make decisions,
plan movements,
execute actions safely.
Autonomy does not necessarily mean complete independence.
Many systems operate only in certain areas, at specific speeds or under clearly defined environmental conditions.
What Does Mobility Mean in This Context?
Mobility includes all systems that move people, goods or machines.
Examples include:
passenger cars,
commercial vehicles,
buses,
trains,
autonomous shuttles,
mobile robots,
delivery vehicles,
drones,
agricultural machinery,
construction equipment,
ships,
automated guided vehicles.
Autonomous mobility is therefore much broader than the self-driving car.
Autonomous mobile robots move through factories. Driverless systems transport goods in warehouses. Agricultural machines operate independently in fields. Autonomous haulage vehicles work in controlled mining and port environments.
How Does an Autonomous System Perceive Its Environment?
Autonomous systems combine several sensor types.
Cameras
Cameras provide detailed visual information. They can detect:
lanes,
traffic signs,
people,
vehicles,
obstacles,
traffic lights,
surfaces.
Machine vision and deep learning allow objects to be classified and tracked.
Radar
Radar measures distance and relative speed. It works in darkness and is often more robust than cameras in rain, fog or dust.
LiDAR
LiDAR creates three-dimensional point clouds of the environment. This allows objects, distances and spatial structures to be measured precisely.
Ultrasound
Ultrasonic sensors measure short distances and are commonly used for parking, manoeuvring and low-speed robotic applications.
Inertial Sensors
Accelerometers and gyroscopes measure movement and orientation.
They are particularly important when satellite signals are temporarily unavailable.
Satellite Navigation
GPS and other satellite systems provide global position information.
High-precision applications may use correction data and additional reference systems.
Wheel-Speed and Position Sensors
These sensors measure movement, speed and steering angle.
Why Is Sensor Fusion Essential?
No single sensor is reliable under all conditions.
Cameras provide detailed information but may struggle with glare or darkness. Radar works in poor visibility but provides less visual detail. LiDAR delivers precise geometry but requires additional processing.
Sensor fusion combines these different data sources.
The result is a more robust model of the environment.
A system may determine that a camera and a LiDAR sensor are observing the same object, while radar provides its speed.
Localisation and Mapping
An autonomous system must know where it is.
This is more complex than simply reading a GPS position.
The system combines:
satellite navigation,
digital maps,
landmarks,
camera images,
LiDAR data,
wheel speed,
inertial measurements.
An important method is Simultaneous Localisation and Mapping, commonly known as SLAM.
With SLAM, a system creates or updates a map of its surroundings while determining its own position within that map.
SLAM is especially important for mobile robots, drones and vehicles operating where satellite reception is unreliable.
Perception and Object Detection
After collecting the data, the system must understand what is present in its environment.
This includes:
object detection,
classification,
tracking,
free-space detection,
motion analysis,
behaviour recognition.
An autonomous vehicle must do more than recognise that an object is ahead. It must determine whether that object is another vehicle, a pedestrian, an animal or a stationary obstacle.
It must also estimate how the object is likely to move.
Planning and Decision-Making
Based on perception, the system plans its behaviour.
Planning often takes place at several levels.
Strategic Planning
The system selects an overall route.
Behaviour Planning
It decides whether to:
maintain the lane,
change lanes,
stop,
overtake,
avoid an obstacle,
reduce speed.
Motion Planning
The system calculates a specific trajectory.
This defines how the vehicle or robot should move during the next few seconds.
Control
Steering, propulsion and braking execute the planned motion.
What Role Does Artificial Intelligence Play?
Artificial intelligence is used mainly in perception and situational assessment.
Neural networks can:
detect objects,
identify lanes,
determine free space,
analyse motion patterns,
classify traffic situations,
detect anomalies.
AI, however, is only one part of the complete system.
Safety-critical applications often combine learning methods with conventional control, planning and monitoring algorithms.
This hybrid approach combines the flexibility of AI with understandable technical safety mechanisms.
Edge AI in Autonomous Systems
Autonomous systems must respond quickly. Decisions about braking, avoidance or stopping cannot be transferred entirely to a remote cloud.
Core processing therefore takes place locally inside the vehicle or robot.
Edge-AI hardware processes sensor streams in real time and executes trained models directly.
The cloud can support:
model training,
map updates,
fleet analysis,
software distribution,
long-term data analysis.
Immediate driving and motion decisions remain local.
Levels of Autonomy
Autonomy is not a single condition. Systems can take over different amounts of responsibility.
A basic assistance system supports only individual functions. More advanced systems may control steering and speed at the same time.
Highly automated systems can drive independently in certain situations but still require defined handover procedures.
Fully autonomous systems are intended to perform all driving tasks independently within their specified operating area.
A key concept is the Operational Design Domain.
This describes the conditions under which the system is allowed to operate.
These conditions may include:
road type,
speed,
weather,
time of day,
geographical area,
traffic conditions.
Typical Applications
Autonomous Passenger Cars
Self-driving vehicles are intended to transport people safely and efficiently.
Robotaxis and Shuttles
These systems often operate in clearly defined areas and on fixed routes.
Logistics and Warehousing
Autonomous mobile robots transport materials inside buildings.
Delivery Robots
Small vehicles carry goods on pavements, industrial sites or campuses.
Agriculture
Autonomous tractors and field robots perform seeding, crop care, harvesting and monitoring.
Construction and Mining
Driverless machines transport materials in controlled environments.
Drones
Drones are used for inspection, surveying, logistics and monitoring.
Shipping
Autonomous or partially autonomous vessels support navigation, port operations and freight transport.
Safety as a Central Challenge
An autonomous system must remain safe in unexpected situations.
These may include:
sensor failures,
hidden objects,
poor visibility,
incorrect maps,
unexpected behaviour by other road users,
communication failures,
technical defects.
Safety is therefore implemented at several levels.
Redundancy
Multiple sensors or computing systems perform similar tasks.
Plausibility Checks
The system compares results from different sources.
Fault Detection
Components monitor each other.
Safe State
In the event of a critical fault, the system must stop in a controlled manner or perform another safe action.
Simulation and Testing
Autonomous systems are evaluated in simulations, on test tracks and in real-world operation.
Cybersecurity
Connected autonomous systems can become targets for cyberattacks.
The following areas require particular protection:
vehicle communication,
software updates,
sensor and control data,
external interfaces,
map and position data.
An attacker must not be able to inject false sensor data or manipulate control commands.
Secure boot processes, encrypted communication and signed updates are therefore essential parts of the architecture.
Communication with Infrastructure and Other Vehicles
Autonomous systems can exchange information with other vehicles and surrounding infrastructure.
This communication may provide information that cannot be seen directly by onboard sensors.
Examples include:
accident warnings,
traffic-light phases,
roadwork information,
road conditions,
emergency-vehicle positions,
movements of other vehicles.
This external information improves situational awareness but should not be the sole basis of a safety-critical decision.
Challenges in Everyday Environments
Autonomous systems work particularly well in structured and controlled environments.
Open environments with many unpredictable influences are more difficult.
These include:
complex city centres,
construction sites,
unusual road layouts,
poor weather,
non-standard behaviour,
unclear lane markings.
One of the greatest challenges is therefore not the typical situation, but the rare edge case.
Sustainability and Efficiency
Autonomous mobility may contribute to more efficient transport.
Potential benefits include:
smoother driving,
better utilisation,
fewer empty journeys,
optimised routes,
shorter waiting times,
better coordinated logistics.
Whether environmental benefits actually result depends on the overall system.
Greater convenience may also create additional traffic. Transport planning, vehicle occupancy and integration with public transport are therefore important.
Conclusion
Autonomous systems combine sensing, artificial intelligence, localisation, planning and control.
They must not only capture their surroundings but also understand them, predict developments and act safely.
Mobility applications range from autonomous passenger vehicles and mobile robots to drones, agricultural machinery and ships.
Technical progress is significant, but reliable autonomy requires more than powerful AI models. Robust sensor fusion, clearly defined operating limits, functional safety, cybersecurity and extensive testing are equally important.
