Robotics and perception in agriculture: smart farming getting smarter

The National Robotarium has been set up to help bring to the field the UK’s inherent R&D expertise in robotics, Associate Professor Dr. Fernando Auat Cheein sets out its plans.

Agricultural processes worldwide are experiencing fundamental changes motivated by different technology revolutions. Examples include the improvement of connectivity resources (such as 5G or 6G) and the Internet of Things with its inherent capability of connecting devices, such machines, into existing process. In this scenario, we see farming practices changing and adopting different (not necessarily new) technological resources that go beyond the precision agriculture as known no more than 10 years ago. We can find drones acquiring data through specific cameras and generating maps; electrical tractors decreasing the carbon footprint and being able to be connected to the grid; robots manipulating, pruning, moving assets; digital platforms for data handling, data management and for enhancement of the decision-making process, among many other examples.

Such technology jargon is motivated by the fact that processes must be greener if we want a healthier soil, a more efficient production, a more profitable industry, or to decrease chemical applications, among many other expectations. What’s more, in most developed countries (and many third world countries as well) we are experiencing a lack of human labour force, which is ironic considering that world population is increasing. This situation represents an ever-present challenge.

One initiative this has spawned is the National Robotarium. This is a partnership between Heriot-Watt University and the University of Edinburgh which combines Heriot-Watt’s engineering heritage and strengths in robotics for hazardous environments, manufacturing, healthcare, and human-robot interaction with the University of Edinburgh’s expertise in space, construction, and humanoid robotics.​ The National Robotarium facilities are based on the Heriot-Watt University campus, in Edinburgh. And within the National Robotarium and the School of Engineering and Physical Sciences (EPS), we are starting the Field Robotics Group, focused on robotics and perception for agricultural purposes.

Figure 1- The National Robotarium at Heriot-Watt University, Edinburgh campus.

I started the Field Robotics Group when I  joined the EPS, UK and the National Robotarium, as associate professor. . The group gathers several PhD students (from UK, Chile and Italy) under Fernando’s supervision, as well as several master students, and colleagues from the HWU. The mission of the group is to consolidate a task force to offer Scottish and UK farmers specific solutions to specific problems in the agricultural sector. The group is supported by the strong collaboration network we’ve created over the past decade, which includes researchers and institutions from UK, Italy, Spain, Poland, Portugal, USA, Latin America (Brazil, Chile and Argentina) and most recently, Greece, China and Turkey.

The Field Robotics Group is currently tackling the following themes, in which their projects are associated with:

Plant phenotyping and characterisation. Using recent advances on artificial intelligence and sensor technology, we are currently able to monitor a crop’s water footprint, detect and classify weeds, computationally model crop canopy, detect disease (especially on tomato plants but this extendsto other crops), and more than 24 vegetation indices that provide health information regarding the crop.

Our work is limited to gathering and processing information, and offers farmers guidelines on how to proceed, but the decision remains on the farmer’s side. A clear example is what we are doing to species monitoring, in which we classify the different species, automatically. We can retrieve the moisture content, height and width of the canopy, the leaf area index, both from drones and from the ground. One of the most important outcomes of our applications is that right now we can predict the amount to be harvested with an error less than 10% on different crops, including apples, avocados, lemons, cherries, grapes, and others. To this end, we use low-cost sensors connected to our algorithms (usually most applications regarding harvesting estimation use cameras from mobile phones). We have validated our work in Spain, California, and Chile, and we are currently exploring partners to apply our knowledge in the UK.

Apple detection and harvesting estimation, a joint work developed with Spanish institutions, using low-cost sensors.

Cherry detection, size classification and counting, work developed in Chile.

Robotics solutions and deployment. One of the fundamental parts of our work is the study, design, and implementation of new robotic solutions for the farming process. There may come a point in the future when robots will be able to do everything on the farm. But right now, they are limited to specific tasks, such as pruning, irrigation, herbicide and fertiliser management, fruit manipulation and interaction with field workers.

Right now, in our group we are facing the micro-fertilisation problem: spraying at a leaf level, and not the entire canopy, thus protecting the plant and the soil. To do this, we are currently designing a new robotic solution that could work under the UK farming weather and according to our regulations and certifications. Although the system is still being developed, we are confident that our solution will highly benefit farmers.

We are also starting a new line of research and development, purely based on improving the potato harvesting and potato plant monitoring, using ground technology. Although we are focusing our efforts on autonomous machinery, autonomy is not yet ready, worldwide, from an energy management point of view: when using electrically powered machinery, their performance is stillbelow that of combustion engines. So we are continuing our research line in improving the power consumption of electrical machinery in the agricultural context.

To achieve that, we are using commercial vehicles for proposing new energy prediction strategies: the power autonomy as provided by the manufacturer changes according to the weather and environment conditions, including the type of terrain, and our work is focused on a more realistic, efficient approach. Our field results so far have shown that we are able to predict the energy consumption up to 30% more accurately than when using the manufacturer’s specifications, and therefore farmers can plan their activities with more realistic costs and data. Our findings can be extended to any kind of electric vehicle.

Vehicle used for modeling the energy consumption in electric vehicles.

Technology surveillance. Our commitment as a group is to be prepared to the UK farmers needs. As part of our’s work at the National Robotarium, the Field Robotics Group is constantly seeking for new solutions and evaluating the feasibility of existing ones. Farmers and agricultural industries are welcome to engage with us in such activities. One of our main goals is to be able to provide recommendations to specific needs.

The new agricultural initiatives at the National Robotarium open opportunities to farmers and to the agricultural industry and there is an invitation to approach, since problems need to come from the farmers side, from the sector that seeks innovation.

Dr. Fernando Auat Cheein is an Associate Professor in Robotics and Autonomous Systems, UK National Robotarium, Edinburgh Centre for Robotics School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, Scotland. He has published more than 100 journal articles, several conference papers and applied (and granted) several patents. He is the author of two books and editor of another two. He is associate editor of Computers in Electronics in Agriculture, Biosystems Engineering (both Q1 journals in the field), Robotica-Cambridge, Journal of Field Robotics and IEEE OJ on Intelligent Transportation Systems. He is also technical editor of ASME/IEEE Transactions on Mechatronics and IEEE Senior Member since 2021. His graduates are academics or researchers in several universities around the world, including University of Lincoln (UK), Carnegie Mellon University (USA), UC Davis (USA) and several universities from Chile and Ecuador. His innovation interests are: robotics/mechatronics and perception in agriculture, electrically powered machinery, automated process, fruit packaging and manipulation, crop monitoring. (f.auat@hw.ac.uk)