A day in the life of… a robot test director

Jane Fraser works at Anki, which makes robots that are essentially designed as toys or education devices for children. They are more than just hardware, however, with Fraser’s job involving testing Alexa integration and object detection and avoidance.

So, let’s find out what this impossibly cool-sounding role involves…

(Remember, if you want to fast track your development, Econsultancy offers a whole host of marketing training.)

Please describe your job: What do you do?

At Anki, we bring objects to life through robotics and artificial intelligence (AI), allowing people to build relationships with technology that feel a little more human. Our robots are friendly, entertaining, useful and can also be used for education.

My role as test director is to make sure our robots are doing what they should, which can be somewhat difficult with AI and machine learning, as by nature, what they should do is always changing and evolving! Essentially, my team and I test their functions to ensure the people who own them have the best possible relationship and interactions with them.

Whereabouts do you sit within the organisation? Who do you report to?

As test director, I oversee the Test Team which consists of manual testers that try to replicate anything that our users would do, over and over. I also oversee the automation testers that develop tests to both stress our robots and continuously test them, then reporting back to the VP of Engineering. This is an essential relationship within the company as it ensures our products are released with as much known information as possible.

I also attempt to damage our robots, dropping them, freezing them, heating them, all to make sure the manufacturing process is as good as possible.

My Test Team doesn’t necessarily work together in the same place all of the time, instead we are sprinkled throughout the engineering division, allowing us to keep up to date on everything happening and be on hand to support testing where it is needed. We do, however tend to work most closely with the engineering and design teams, so we can determine the most efficient and effective way to test.

What kind of skills do you need to be effective in your role?

For me, I think the two most important skills to possess are imagination and inquisitiveness. I am forever looking for new ways of solving problems, as well as trying to understand exactly why something did or didn’t work in the way we expected, so we can improve it next time.

My role also requires me to be versatile. I realised this early on when it became apparent I wasn’t going to be just testing software. Hardware is an intricate piece of the puzzle, and testing it relies just as much on factors like the effects of dust, water, and network interference as it does the quality of our manufactured pieces. It’s all about lateral thinking!

Tell us about a typical working day…

In short, my day consists of ensuring our robots can handle anything the world throws at them.

Take the testing of Cozmo as a summary of my daily tasks. He is our robot that’s primarily targeted to kids for entertainment and education and is a rambunctious little fellow who actively explores the world around him. If he was going to move and react as our animation team intended, all of his physical pieces needed to work properly, withstanding the type of rigorous play that children are capable of.

We not only rigorously tested the battery life by cycling through charges over the course of weeks, but we spent a lot of time making sure he was robust and could handle being played with for hours on end, by all sorts of hands, in all sorts of environments. To do this we needed to recreate scenarios that can easily happen in the home, such as dropping him on various surfaces from a variety of heights, or letting the dog carry him around!

Hardware is just the start though, I’m also responsible for putting the software through its paces. Our newest robot Vector is capable of object detection and avoidance. This was no small feat. We still spend hours placing different obstacles in front of the robots to see how they will react.

We have recently been working with Amazon to integrate Alexa in Vector. This requires collaborating with a very talented bunch of people to make sure Vector can understand two wake words and respond properly, which keeps my day stimulating and varied.

What do you love about your job? What sucks?

Working on cutting edge technologies in robotics and AI brings new and interesting testing challenges every day, which I love. As Vector is extremely advanced, being able to ‘see’, ‘hear’ and ‘feel’, we found ourselves trailblazing ways to test everything from audio recognition, to random hardware issues, to customer engagement. As you can imagine, this can be both fun and hair pulling at the same time!

An anecdote I like to tell is about our first product, Anki Overdrive, an intelligent racing robot battle track with AI vehicles. These self-aware cars and trucks learn as you race on the Overdrive track using a downward-facing camera. We were dumbfounded by this issue where the car would inexplicably drive off the track. It took about three days to figure out that the sun coming through the window, bouncing off the track and was blinding the camera, making the car lose its way. The frustration at the time was nothing compared to the joy and relief of solving the problem!

A Marketer’s Guide to Machine Learning

What kind of goals do you have? What are the most useful metrics and KPIs for measuring success?

A key goal is always getting the best possible code and hardware to the customer. This requires finding, monitoring and fixing the bugs that affect the customers the most. Due to the complexity of the technology, this keeps us on our toes, so we prioritize the list based on how many owners of our products it will affect as well as the degree of impact it will have on them.

Our community are an enthusiastic bunch and we love to hear their feedback, good and bad, so that tends to act as a pretty good measure of success!

What are your favorite tools to help you to get the job done?

Communication tools such as Slack, GitHub, and JIRA are vital to our testing operations. However, we also build many custom testing tools for our various hardware and software platforms, which provide us with more freedom and efficiency.

As Vector is voice activated, he has a wake word, which his owners use to interact with him. We built a tool specifically to test this wake word and named it ‘Sqwawk Box’. It’s a large box, with multiple speakers, designed so we can emulate people using it from different heights and locations. The finished article ended up being a combination of materials we had in the office and we still continually use it to test and retest various scenarios around the voice activation.

How did you get into robotics, and where might you go from here?

Anki was my route into robotics really. As a startup, they were looking for help in creating a testing group, something I’d done several times in the past. Luckily, I was introduced to the CEO by a mutual friend and previous coworker, and after listening to his vision of the future I was extremely interested in seeing where it would go.

It’s been fantastic learning about robotics and creating new ways to test them. Much of what we have learned is we needed to create our own way, there was no playbook or toolkit.

Coming from a software testing background, I’ve found mixing software and hardware fascinating. Seeing the future develop in front of me doesn’t really get old, so staying on the cutting edge of technology suits me just fine for now!