GROWBOTICS 2017

In 2017, during my PhD at King’s College London, I was awarded a grant from Science Gallery London to develop a public engagement activity related to my research on automation for the horticulture industry, through the Collaborate & Engage public engagement course.

This course took 12 selected KCL PhD students from Arts & Humanities and 12 PhD students from the science faculties, and put them through a 5-part course on public engagement. As part of the course there was funding for up to 5 projects with everyone submitting a project proposal, and fortunately my Growbotics project made the cut!

In addition to the support from Science Gallery London, I made contact with the production manager at Vitacress Ltd, Chris Moncrieff, who I had previously spoken with about automation in Horticulture and Agriculture and who I knew was a big advocate for the research ongoing at King's. Chris kindly organised sponsorship from Vitacress Ltd. to help further support these workshops, many thanks Chris!

Planning

This has been an exciting process - working with me are a team of three other researchers from the course (Peter Gordon, Elizabeth Mann, and Alison Skipper), and two artists (Amina Nazari and Dean Rogers) who have been brought in to the project due to their expertise in producing interactive exhibits and events. Without spoiling the surprise, we'll be bringing a (hopefully) very cool interactive exhibit which explores the themes of my research later this year to an exhibit space in central London.

As part of this Collaborate & Engage project, there was scope for a more focused public engagement activity which led up to the main interactive event. Originally this was planned to just be a series of talks in local schools; however through a fortunate introduction to the KCL Robotics Society president Arthur Zargaryan, we hatched a plan for the Growbotics Workshop series. The goals for the workshop were identified early-on as:

  1. Teach the students something new.

  2. Encourage students, who are making decisions on their future education direction, toward STEM (leading to a target age group of ~16 year olds).

  3. Specifically encourage these students towards Robotics.

  4. Encourage them to consider the application of robotics to areas they might not have previously considered, i.e. promote research in Horticultural/Agricultural automation.

An overarching message for the workshop series was that automation through robotics and AI is playing an increasingly important role in society, and that the skills learned through working on robotics projects equips you with a wide scope of knowledge to address a large variety of problems.

Students were selected from local schools with help from KCL Marketing. The students' own experience in programming or robotics was not a factor in their selection, it all came down to how enthusiastic they were!

The workshop was spread over four days, with the first two days focused on teaching, and the second two days focused on the students applying their new skills to a grand challenge.

Getting started - Day 1

To introduce the students to the workshops, and more generally to robotics, I opened the day with a presentation describing some different varieties of robots and explaining the trade-off between the level of structure in a robot operating environment and the level of autonomy that the robot can achieve given this structure.

Examples of this trade-off can be seen in welding robots which operate in a highly structured environment where the positions of all objects it interacts with are known in great detail, and so can operate autonomously; versus a surgical robot working in a human body is operating in a highly uncertain environment and so must be largely teleoperated by a human surgeon. Understanding this trade-off helps give students better awareness of what makes robotics difficult, and to understand the drivers behind current research directions in robotics.

The robot platform of choice for the workshops was the Lego Mindstorms NXT. We used the  LeJOS firmware on the robots, which let the students program the robots using Java. Given that only one or two of the students had any experience with Java, Aliin and Maria from the KCL Robotics Society kicked off the workshop with a Java crash course where the students learned about variables, control flow, methods, and even a bit about classes. There was quite a lot to cover in getting the students up to speed with Java, so for the first day we focused on purely code-based mini challenges (print prime numbers up to an input value sort of problems).








Building skills & knowledge - Day 2

The introduction for the second day aimed to further develop the students' understanding of research in robotics, where I described specifics of my own research in programming robots to perform horticultural tasks by human demonstration. After going through my research background, I gave a short talk on how you can use Finite State Machines (FSMs) to help simplify control code design (or at least help you reason out your programming approach when faced with a new problem). The emphasis here was to encourage the students to get into the habit of planning out their code when attempting to solve new challenges to help them spot pitfalls, and maintain a manageable structure in their code.

After the opening talks, it was time to break out the robots! The students were tasked with the challenge of creating line following robots using a FSM way of thinking. KCL Robotics did a great job of creating an easy to use API for interfacing with LeJOS that helped the students get their robots up and running rapidly.

In order to help the students better control their robots, Arthur from KCL Robotics Society gave the students an introduction to robot kinematics, explaining how to use kinematics to convert rotations into linear distances. Many of the students really enjoyed this portion of the workshops, as they were challenged with applying some math they were familiar with from school in a way that helped them better control their robot in the real-world.

We were very lucky in the timing of the workshops as during our lunch breaks we were able to visit the KCL Festival of Science, and the KCL NMS Impact Showcase. Here, students had the chance to interact with researchers working on a wide variety of problems and speak with them about their work first-hand. Highlights from these events included exploring virtual reality sound systems, interacting with EMG controlled robot hands, and seeing the fun projects ongoing in the Wheatstone lab maker space.

Putting skills into action - Day 3 & 4

Once we had the students up to speed with Java and the LeJOS API, it was time to introduce the main challenges to them. They were split up into teams of 6, with 3 members of the team working on the first challenge, and the other 3 members working on the second challenge (and ultimately all 6 members working together toward a fully automated system).

The first challenge was the Yardbot - a robot challenge based on the pot-moving robots developed by Harvest Automation, where students had to develop a robot capable of picking up Lego plants and arrange them on a marked out grid.

This extended the work they had been doing on line-following robots, but required them to overcome additional challenges such as crossing over lines, maintaining some kind of memory of where the robot was in the world, and working out successful strategies or build modifications for actually grabbing the Lego plants with their gripper.

As can be seen in the video below, they more or less managed to get there in the end! I was really impressed with all the students in this challenge, as they explored not only various software-based solutions to the problem, but also explored how to vary the robot's embodiment to better suit the problem - one group even ditched the pincer gripper design we provided all together and opted to build a forklift!

The sensors and LEDs are connected to an Arduino Mega, which is in turn connected to a computer via ROS - you can find the source code for the project over on my github (Arduino/ROS code and GUI code). The LeJOS API can be found on the KCL Robotics Society Github here.

The second challenge was the Vertical Farmbot - a robot challenge based on the automation of a vertical farm.

In case you aren't familiar with the idea, vertical farming involves growing plants on shelves, greatly improving land-use and potentially the growing conditions. These can range from very basic home garden setups, to advanced commercial systems which grow the plants under artificial light in environmentally controlled conditions and replace soil with liquid nutrient baths. The benefits of vertical farming attracting industry interest include reducing soil-use, improved pest resistance (in turn requiring less use of pesticides), and improved energy efficiency (easier to maintain temperature conditions in an insulated warehouse than a glasshouse). It's a growing method with a lot of potential and it has already seen some commercial success. For the workshop challenge, the students are tasked with taking the next step and figuring out how to fully automate a vertical (Lego) farm!

Arthur got to work in the Wheatstone Lab to laser cut the shelves, designed so that we could smoothly integrate RGB lights and light-dependent resistors (LDRs). There were 16 RGB lights, originally meant to be used to represent the growth-stage of the plant under it, and 16 LDRs, used to detect the presence of a plant for our scoring system, I set about wiring up and programming the lights and sensors to allow us to monitor and control the system state during the Vertical Farmbot challenge, along with a user interface to help us keep track of scores as students picked up the plants.

original_challenge.png

Originally, taking inspiration from Google Hash Code style problems, the challenge was an optimisation problem. Given a vertical farm with an unknown number of plants occupying the shelves, the students had to scan each shelf to first determine the occupancy of the system, and then determine each plant's age.

The plant's age was determined by an RGB light downlighting it, with blue indicating the plant is too young, green indicating it is just right, and red indicating it is too old.

The students would then get 5 points for each green plant delivered to the goal, 1 point for every red plant delivered to the bin, and -1 point for every blue plant collected. The goal was to maximise their score in a given amount of time.

It quickly became apparent that this was a bit too much for two days of work considering many of the students had just started programming at the start of the workshops, so we revised the challenge to focus on the subtasks needed to successfully take items of the shelf and deliver them to a goal location.

  1. Horizontal detection - Build up a map of one shelf, returning a 1D array of 1's and 0's indicating if a plant is in a position.

  2. Multi-level detection - Build up a map of all of the shelves, returning a 2D array of 1's and 0's indicating if a plant is in a position.

  3. Positioning - Write a function which takes an (x,y) position and move the crane arm to this grid location on the shelves.

  4. Plant picking - Write a function which uses the crane system's arm to pick up a plant, and then place it back down.

  5. Plant drop-off - Extend subtask 4 to drop-off the collected plant at a goal location.

  6. Put it all together - Use all of the subtasks to automate the vertical farm and collect all of the plants!

The Vertical Farmbot proved to be a pretty challenging task for the students; however in the end they managed to get there, picking up a Lego plant from a given position and bringing it to the drop-off location. See the video below for some clips of the students' attempts and close-ups of the crane in action.

Reflections & Plans

Looking back on the workshops, it was a really fun and rewarding experience! There was a significant amount of preparation and admin needed to pull it all together, and some stressful moments, but overall the workshop series was a success, judging from very positive feedback from the students.

It definitely wouldn't have been possible without the help of KCL Robotics Society, KCL Marketing, KCL Student Union, and the workshop sponsors Science Gallery and Vitacress! A big thank you to the teaching assistants Aliin, Maria, Vikash, Genki, Luca, Asif, Andrei, Jianwei, and Henri. A special thanks to Arthur for putting in a huge amount of work helping build the Vertical Farmbot! You can find a write-up of the Vertical Farmbot development over on Arthur's website as well! And of course, many thanks to my research sponsors AHDB.

I hope to bring back the workshops bigger and better next year, but in the meantime I will be working with my project team to pull together our main event for later this year - watch this space!

Finally, many thanks and well done to all of the students that came along for the workshop, you all worked hard and did a great job!

Student Feedback

Good workshop, I was happy about having someone always there to help.

Absolute fun! Thank you for inviting me. It was educational and exciting.

Great to see the way robots would work in horticulture, it was really interesting testing it out ourselves.

It was perfect, there is something for everyone.

Feedback

Slides

Code

https://github.com/Robotics-KCL/LEGO-API

Contact Details

Any questions, please contact me at aran[dot]sena[at]kcl[dot]ac[dot]uk

Growbotics Project Team

  • Aran Sena

  • Arthur Zargaryan

  • Peter Gordon

  • Elizabeth Mann

  • Amina Nazari

  • Dean Rogers

  • Alison Skipper

Collaborators

This project is run in collaboration with: 

KCL Robotics:

Amina Nazari: 

Dean Rogers: 

Sponsors

A special thank you to our sponsors, Science Gallery London & Vitacress Ltd!