msadowski blog

Build Toolkits

2025-05-20T00:00:00-00:00

Recently, I’ve been thinking about some good practices in robotics software development and realized that one of the most impactful is the one I learned early on. The lesson is: Build Toolkit(s).

Background

In 2014, I wrapped my degree and moved to the UK to join SkyCircuits (now Callen-Lenz), a company that at the time was developing high-end autopilot for drones. What set us apart at the time was a modular control architecture, on-board script support, and out-of-the-box setup for any platform (multirotors, fixed-wings, helicopters).

We built three user-facing pieces of Ground Control Station (GCS) tools:

Plan - mission planning
Flight - mission execution
Toolkit - a power-user interface to the autopilot

As you probably guessed from the title, the reason we’re here is the last item on the list.

SkyCircuits Toolkit

SkyCircuits Toolkit was a power user interface that allowed plotting data in real-time (super handy when tuning PIDs) and updating any variable or script on the autopilot. Very useful for all the teams within the company and any power-users that needed it.

I didn’t realize it at the time, but this was a lesson that stuck with me. And ‘Toolkit’ is such a great name that I always tend to use it.

“… and we will build a power-user interface exposing all our parameters, and we are going to call it Toolkit”

Me, on at least two occasions with two different clients

Create software for power-users

Many electromechanical systems require a unique set of parameters that you need to set for each device that you build, no matter how much cookie-cutter-like is your approach. The configuration you may store are PID controller terms, IMU offsets, camera calibration values, device names and so on.

Having worked with over 50 companies, I’ve repeatedly seen software developers involved in provisioning new devices and tuning parameters by hand. This approach is OK when you are prototyping, but once you start scaling up, this becomes an issue.

To illustrate, let’s say the PID controller values of your autopilot are stored in a database, which is the only interface for changing these values. In such a situation, the following request would not be unheard of:

Hey Mat, we changed the ESCs on our multirotor and now it’s a little bit wobbly in hover. Can you tune it?

A simple request like this can cost you 30 minutes at best (that includes the context switching cost). At worst, it might require a field trip and take half a day or more. That’s time your developer isn’t contributing to the product.

By creating power-user interfaces, you can level your robot set-up game between your software, mechanical and electrical teams. Asking a mechanical engineer to learn ROS 2, SQL, or configuration-storage-solution-du-jour is a big ask. Abstracting these interactions into a piece of software that remains stable regardless of the underlying implementation will make all your team equally productive when it comes to robot set-up.

What’s the next step after building your Toolkit? Naturally, automating the entire provisioning process!

Build Toolkits was originally published by Mateusz Sadowski at msadowski blog on May 20, 2025.

Remote Robotics Consulting - Seven Years In

2025-04-23T00:00:00-00:00

Intro

If you’ve been following my blog for a while, you might have seen my earlier blog posts summarizing my experience as a Remote Robotics Consultant. In case you’d like to start from the beginning, here is the ordered list of my posts:

In hindsight, there are some choices that I made that turned out suboptimal, but if I turned back time, I’m not sure if I would be able to make better decisions. Let’s see if there are some lessons to be learned!

The section below describes my last long-term contract, if you’d like to jump into some meaty advice, see the next section, Lessons Learned.

Update on my Previous Blog Post

In my last consulting update, I ended my blog post with a bit of a teaser of what is next and in my increased involvement with DTE, an Icelandic company building laser-shooting robots for molten metal analysis. I considered the company’s product the pinnacle of tackling DDD (Dirty, Dull, Dangerous). The smelters themselves and metal processing in general are no joke, and people get injured in these places very often.

Do you take the leap? - My thinking at the time

Here was my thinking over two years ago - if I was given a choice to join a company I believe in, should I drop my consulting operation, and go full in?

It turned out dropping consulting was not necessary. The alternative was going through an intermediate company. The math of becoming an employee of an umbrella company was quite interesting, I would double my hours, but because of all the taxes and fees involved, I would end up earning 100 USD more a month.

Given this, we decided I would remain a consultant, with all the perks that come with a job description (no fixed schedule, can take other clients) and on top of that, receiving stock options, and having a three-month notice period.

One discussion I will always cherish from my time at DTE was when I was chatting with my manager at the time, Lúðvík, and the conversation went along the lines of:

“Sure, I can lead the robotics team, but it might be difficult since I’m not working full-time” (was at 80 hours per month back then)
“What? Based on your output, I thought you were full-time”

This is me

I indeed felt like I’ve reached my peak productivity with that company.

As I’m publishing this blog post, I wrapped up my cooperation with DTE. It’s been an interesting and very intense period of my life, and as a result, I can tell people that I’ve worked on heavy metal robots that shot lasers.

Lessons Learned

Of course, there was going to be a lessons learned section. Today, I have for you a mix of business and technical advice:

Retainers are amazing - it feels great to lock in the hours at the beginning of the month. If you go this direction, consider how to handle under/over time (charge for it at the beginning of the next month, shift hours around, take them on as holiday)
If you are entering a long-term contract, having a notice period in the contract might be a good idea for both parties (in my case, it gave me three months of runway to have a think about what’s next)
Think of your insurance - metal smelters are very dangerous, which meant that if I was to go to one on behalf of DTE, I would not be covered by their insurance (since I’m also external to DTE). Always consider these in any hazardous environment!
Eliminate repetitive busy work as soon as possible - developers should develop, not perform mundane, repetitive tasks. If this is the case, automate the tasks or create tools to reduce the barrier for performing these tasks
If you don’t have time to create content for your main service, then what are you even doing? - this one stings a bit. When I went all-in with the long-term contract, I didn’t have time or energy for side explorations, and this blog has been dormant. In hindsight, I should’ve taken fewer hours and played more

Upwork

In every single of my updates, I mentioned Upwork, so will do it here too. It feels that Upwork ended when they disabled their RSS feeds.

For the past three months, I’ve been browsing the jobs in ROS, robotics and drone categories, and the quality of jobs went downhill from the last time I was actively using the platform. Back in the day, you would see 1-3 quality projects a month, and in the past three months… nothing. They are also upping their fees again to a variable 0-15% (want to bet it will be closer to 15%?)

Upwork is heavily pushing their AI tools for proposal writing, and I think AI will be doing a big disservice to the platform. When I created a job un Upwork the other day, I had a couple of long-form proposals within a minute of posting it. Clearly, AI bots are hard at work on Upwork, and I don’t think this is helping them.

Weekly Robotics

In June last year, I took a sabbatical from my newsletter Weekly Robotics. At the time, I was focusing on reconstructing our apartment, and preparing to welcome my daughter into the world. I was also hoping to finish Baldur’s Gate 3 but didn’t have many chances to play since that time.

Today, the newsletter is back in strength. I started self-hosting the e-mail infrastructure and automated the shit out of issue creation. At one point, I was thinking if I should use AI to create feature description but decided against it since I’m quite a bit tired of all the AI slop on the Internet.

My goal for the newsletter for 2025? Grow it to 20,000 subscribers across e-mail and LinkedIn!

What’s next

I’m tempted to try out something new, and offer “Senior Robotics Internship package” that as foresee would work as follows:

Fixed three month cooperation
Working on a specific problem the company is facing
Throwing in Weekly Robotics sponsorship as a bonus
Can be paid in money or equity or a mix of thereof

I think it would make for an interesting experiment, If you’d like to try it out, feel free to [email protected]">get in touch!

Until then, we will keep playing, and exploring coverage path planners.

Learning everything there is on coverage path planning

Remote Robotics Consulting - Seven Years In was originally published by Mateusz Sadowski at msadowski blog on April 23, 2025.

A Roboticist Visits FOSDEM 2025

2025-02-06T00:00:00-00:00

Intro

This year, I paid my first visit to FOSDEM (Free and Open Source Software Developers’ European Meeting) and helped organize the Robotics and Simulation Devroom. This blog post summarizes my experience.

Last year, Kimberly McGuire approached me about organizing a robotics devroom at FOSDEM. One thing led to another, and we assembled a dream team of organizers for the “Robotics and Simulation Devroom”:

Organizers of the Robotics and Simulation Devroom

and just like that, I set off for a Club Mate-fueled endeavor where hackers meet open-source—or should I say, make open-source possible.

What sets FOSDEM apart from other conferences I attended in the last couple of years is the fact that the event is free to attend. I think it does not get more open-source than this.

The Robotics and Simulation devroom

Graphics for our room by Michał Kalina

Our room hosted the first Robotics devroom in FOSDEM’s history. In the last section of this blog post, I’ll provide a list of robotics-related talks from previous FOSDEMs. We had a room for a half-day and managed to squeeze in 14 talks in less than four hours. This is largely due to many presenters agreeing to turn their talks into lightning talks. The room was full for most of the time, with people occasionally queuing outside to enter.

The talks I found interesting

Below is a short list of talks that I found interesting and attended:

The whole Robotics and Simulation track
SatNOGS-COMMS: An Open-Source Communication Subsystem for CubeSats - stepping outside of my comfort zone and going more embedded. Some very interesting lessons learned about using Zephyr-RTOS
The road to open source General Purpose Humanoids with dora-rs - very nice demo with Pollen Robotics Reachy robot and, hopefully, a step towards general robots
Exploring Open Source Dual A/B Update Solutions for Embedded Linux - good overview on strategies for reasonable firmware update workflows and overview on what’s out there
Discovering the Magic Behind OpenTelemetry Instrumentation - trying to expand my horizons on logging. Implementing this with ROS would make a really nice side project
What can PyArrow do for you - Array interchange, storage, compute and transport - I’ve heard of Arrow quite a bit lately and I’m excited to look more into its IPC capabilities and zero-copy approach. Did you know that with Arrow you can seamlessly switch between Numpy, Pandas, and other libraries with zero-copy using Arrow?
Programming ROS 2 with Rust - Julia made a nice presentation in the Rust room on using ROS 2 with Rust
Lessons from rewriting systems software in Rust - Some interesting insights on rewriting software that I believe does not only apply to Rust. One point that drove it home was depending on other’s people code

Reachy used for a demo in the dora-rs presentation

Here is a list of talks I didn’t get to see but that I’ll catch up on once the videos are available:

Tips&Tricks

High Quality Proposals Wanted

Robbie developed by Kimberly to bravely promote our devroom during the event

This year, while organizing the Robotics and Simulation devroom, we received 24 proposals and had to make tough choices to fit them into half a day. We selected only the best and asked some speakers to condense their presentations into 5-minute lightning talks.

Some advice for next year’s applicants:

Make a super high-quality proposal. You are competing for a speaking slot with world-class roboticists, a two-sentence abstract won’t cut it, no matter the reputation of your project
Don’t use AI to write your proposal (reviewers can often tell, and you get huge minus points for that)
It’s always a good idea to submit a link to your repository in the proposal. It’s great if it shows some history of development and it’s not just a handful of commits

Accommodation

I booked the accommodation within walking distance of the venue, thinking it made perfect sense to be as close to it as possible. It turns out the right question to ask yourself is: Are you going to hang out with someone after the event? If you do, where will it happen? Would you prefer to commute early in the morning or at night?

On Friday and Saturday, I spent the evenings near the city center of Brussels with my fellow co-organizers and had to commute back at night towards the campus. In hindsight, I would prefer to sleep closer to the center and get to campus early.

Making it to see the talks

Often, the rooms are packed! If you find yourself going to a talk that might be popular, you get inside the room earlier. Once the rooms are full, you are not supposed to enter them for safety reasons.

Taking notes

When we organized the room, we asked the speakers to upload their slides to Pretalx. This meant that the slides showed up in the talk description and as an attendee, you could download it and make notes right there on the slides. If I just had my tablet with me!

List of robotics-adjacent talks throughout FOSDEM history

2016

Simulating Humanoid Robots in the Cloud: the testing behind the biggest world competition

2017

Loco Positioning: An OpenSource Local Positioning System for robotics

2018

2020

2023

FOSSbot: An open source and open design educational robot (Lightning Talk)

2024

These are all the thoughts I have on FOSDEM 2025. See you in Brussels next year!

A Roboticist Visits FOSDEM 2025 was originally published by Mateusz Sadowski at msadowski blog on February 06, 2025.

Remote Robotics Consulting - A Five-Year Retrospective

2023-08-05T00:00:00-00:00

Intro

In this blog post, I will highlight some of the updates since my last blog and offer some advice that I hope will be useful for anyone looking to get into technical consulting.

Strap in, and let’s go!

Update on my Previous Blog Post

Last time, I mentioned multiple things that are now out of date:

I no longer rent out a workshop. Focusing more and more on software projects, I practically stopped using it. The 30-minute commute one way did not help either, so I resigned from it at the beginning of this year.
Currently, approximately 2% of my projects come from Upwork. They are only short-term consultations. The rest are direct projects with clients.
I solved the context-switching problem - I haven’t had issues with it since my last post, so that’s great news. What helped here was turning off all notifications. I found my e-mail the most distracting and have since decided to check it at my own pace.
Work-life balance? - Here, I fell behind compared to the last update. On two occasions, I was edging on burnout. Things are better now, but I realized I don’t have hobbies outside of robotics. I’m actively working on this and starting to see the light at the end of the tunnel.
In my last update, I mentioned taking on all opportunities that fell into my lap. Well, not anymore. I’ve become more selective and more mindful of my schedule.

Bye-bye workshop.

Passive Income - Here I Come!!

Last year, I released a Manning Live Project series on Building Mobile Robots with ROS 2. Since it’s been released, around 230 students have enrolled in it. This project took me slightly over a man-month of time, invested over six months. Since I’m about to earn my advance, earning a month’s consulting salary from this project will take approximately 19 years. Even though I’m not earning a huge amount from this project, if I could go back in time, I would do it again. It’s been an incredible journey, and I think it’s time to look into publishing more content like this. Lately, I’m finding writing quite rewarding.

Weekly Robotics

Three months after I became a consultant, I started Weekly Robotics, a newsletter that sums up the most exciting research, projects, and news I have come across during my industry research. The newsletter is now closing on 12,000 subscribers on two delivery channels.

Header images of all Weekly Robotics issues in 2022.

2022 was a terrific year for the newsletter, where we had many advertisers, started the first long-term partnership with AMD, and managed to budget about $10k for its growth and newsletter content.

I’ve put this money towards:

Visiting ICRA this year
Getting technical folks to write long-term articles
Doing GitHub sponsorships for some open-source robotics developers
E-mail distribution, hosting, etc.
Subcontracting sponsor outreach
Article editing

The newsletter, however, is far from profitable, especially if I’m not spending this time on consulting. Last time I checked, the time spent on this project annually is about one man-month. It also does not bring in many projects (I’ve counted three since I started it). This is understandable, though, as I’m not advertising my consultancy there and prefer it to be separate from my branding.

The best thing about the newsletter now is the community. Our Patreon slack channel has grown quite a bit, especially since I started inviting non-Patreon users. Having a small community to bounce ideas off of and discuss robotics and projects is great!

Insights

These blog posts are helpful to many (or so I’ve been told), so let’s jump into the meat of it. Technical consulting is not a zero-sum game, so I’m more than comfortable sharing what has worked for me on this journey. I hope some of you will find these points helpful.

My Problem-Solving Strategy

If you have clients who are no experts in robotics, as a consultant, you can have a big leverage on them. Let’s consider an exaggerated situation where a client wants to move an item from point A to B and they know nothing about automation. To drive the point, let’s say that you either:

Propose to develop a ground vehicle with a manipulator arm attached
Tell them the conveyor belt exists and to check them out

Approach a) would provide you with lots of work (and money), while approach b) would either mean that you don’t engage in the project or you take the role of an integrator. Maybe I’m a bit too idealistic, but I always use the latter approach and put my customer’s interest first. I think it’s much easier to work on projects you believe in, and you sleep better at night, too!

Two Rooms

There is a massive overlay between engineering consulting and consulting in the creative fields. In his book, The Business of Expertise, David Baker presents a concept of two rooms that I, of course, took over and incorporated into my strategy.

The two-room concept. Source: davidcbaker.com

In my work, I have two hats:

Consultant - providing high-impact and high-value advice. Usually, this is quite high-level (strategy in the image above)
Contractor/Freelancer - very hands-on role, implementing hardware and software to solve customer problems (execution in the image above)

If you start with a client providing strategy-related work, moving to execution is relatively easy and can be a part of your package. If you begin with execution, it might be tough to start wearing a strategy hat. You’d need a mix of luck, good salesmanship, and someone inside the organization recognizing your value.

Shut Up and Listen

When listening to a client, you might feel that you know how to fix their issue and that it is straightforward. Even though the urge to interrupt might be strong, you should not jump to conclusions. Chances are, your client has worked for years in their industry, and they know it way better than you ever will. So strap in, shut down your eagerness to propose solutions, and ask good questions instead.

Client: … and the UGV we are working on needs to be localized to within 1 meter in the target environment.

You: Easy-peasy, we will take a LiDAR, put some SLAM library on this baby and it’s done.

Client: and the target environment for this client is a mirror factory.

You: …

You, after rudely interrupting a client with your solutions, instead of letting them finish describing their problem in detail…

Don’t Do the Price Dance

Sometimes, you’ll get approached about a project. You will exchange a couple of e-mails, then you’ll take a 30-60 minute unpaid call, then you’ll create a summary of the meeting, and then, you’ll gently start hinting towards talking about money. If you do this, then I hope it works for you. Chances are, you’ve wasted everyone’s time (most importantly, yours).

What I tend to do these days is reply to the first message along the lines of:

Hi!

Thanks for your message. What you describe sounds like a project I could help with. I charge my clients on an hourly basis at $X. If this works for you, I would love to have an introductory call and take it from there.

Quite often, this will be the last interaction with a prospective client. Some people will also get offended if you discuss the price too early. ‘Godspeed’ is what we tell these folks and we move on.

The clients worth working with don’t question my hourly rate and respect my expertise, so it’s always worth being upfront about your fees.

Charge by the Project if You Want to Be Rich.

Jonathan Stark has a blog post and even a book series on how projects, where you bill by deliverables, are superior to hourly-based ones. Working like this is comfortable for your clients because they know how much they will pay upfront, and for you because you’ll incorporate a hefty premium in your estimate, and it will all work out, right?

Well, maybe. I find it e-n-o-r-m-o-u-s-l-y difficult to price R&D projects because it’s all fine and well until you suddenly need to recompile a Kernel of the onboard computer, or it turns out that a fried electronic component caused a bug you’ve been tracking for three days or the customer has changed requirements. Now you need to explain how this eats into the budget and figure out how to charge extra and get paid.

I think fixed-price projects work if you have a good idea of what you are doing. This means you have experience with most of the hardware the client wants, or you are in a position to choose for them, and you know roughly how to solve the software part, and ideally, you have delivered similar software/hardware in the past.

Remote Hardware Work Overhead

Working remotely on hardware (e.g., integrating a LiDAR, IMU, or some SBC) takes 2-3x longer without physical access. Because of that, I usually advise clients to ship hardware to me or get me to their location. For purely remote work, it’s best when the client has a local team that can do the hands-on work and gather data for you, following your instructions.

Don’t Withhold Expertise

Imagine a very theoretical scenario of a conversation between (Y)ou and two consultants (C1, C2) during an introductory call:

Y: We need help with our localization stack. For some reason, our robot position estimate is not turning in the global frame when it’s rotating just fine in real life.

C1: Oh, that's not ideal. I've set up the EKF on hundreds of platforms. If we work together, I'll be able to fix this.

C2: This is an interesting problem. If I were you, I would check which axes are enabled in the EKF and ensure that the yaw axis of IMU or the angular velocity from wheel odometry is fused. If enabled, I would double-check the covariances in your sensor messages.

Which consultant would you pick to work with? Using the second approach has been working well for me so far. I would not worry about a hypothetical scenario of a client taking your advice and running away. It will happen occasionally to C1 and C2.

Even though it might be far from reality, C2 can appear more knowledgeable during the call because C1 didn’t showcase their skills. Having some portfolio can help C1, but after the call, C2 already contributed value to the company, and they might be more keen to hire them.

Consider Regulations

If you have this grand vision of developing the slickest delivery drone or a new robot for eye surgery, and your vision is very clear (pun intended!), and you want it to be on the market in half a year, I might have some bad news. Some fields are highly regulated, and for a good reason. Regarding drones, you should look at FAA certification in the U.S. or EASA in the EU.

My answer in these situations is usually: “Great, let’s do it. I can help you build the prototype, but let’s get onboard someone who knows the regulations so that we start building with these in mind”.

If you disregard the certification, you might create a prototype that works very well. In your tests, it could deliver the packages, but when you start looking into what is required - you might need to change your autopilot, triggering a rewrite of your software. By the way, we need a parachute now, and we won’t get as much flight time, so we need heavier batteries, but this will end up with us being in an even more controlled certification category… and it goes on.

Doing these rounds is undoubtedly fun, and you learn a lot, but, as a client, do you want to keep throwing money into the pit, instead of investing in a clear roadmap?

Be Careful Using Upwork for High-Level Consultations

Upwork is quite a good place for finding projects. In exchange for a 10% fee, you get some discoverability and payment protection for hourly projects. The payment protection will only work though if you move your mouse and press keyboard keys enough during your work. I had this issue with one of my clients - having a very high-level consultation that did not require me to do any input on my computer, and it just so happened that the customer’s credit card bounced.

Upwork then looked through my keyboard/mouse activity during the call and zeroed the activity periods without enough action (even though the app takes screenshots of your screen, and I could be seen on a call), removing a big chunk of my consultation time and never paying me for it. All of this is in line with their T&Cs. After the customer has their account revoked, you can manually add ‘lost time,’ but they can say no, and I don’t think there is much you can do about it, so use Upwork with caution

Fin?

Over the past two years, I’ve worked increasingly with DTE, to the point where I’m well-embedded with their team.

DTE promo video

The company develops solutions (hardware and software) for the analysis of the element composition of alloys. I always explain how the system works by shooting lasers into alloys and using spectroscopy on the light emitted by the process to estimate the element content of alloys.

Me, scaling a volcano in Iceland during one of my many visits to DTE’s headquarters.

As a consultant, I’ve worked with about 50 companies now. There are only three that I would invest in, given a chance. DTE is one of those companies.

Even though I’m on a monthly retainer with one company at the moment, I still consult with others. But being so involved with a company day-to-day, you might be given a choice.

Do you take the leap?

Remote Robotics Consulting - A Five-Year Retrospective was originally published by Mateusz Sadowski at msadowski blog on August 06, 2023.

I Created a Project-based ROS2 Course

2022-07-05T00:00:00-00:00

Earlier this year, I made a beginner’s course Build Mobile Robots with ROS2, that focuses on mobile robots. This blog post describes my experience and thought process when designing it.

Introduction

I used ROS for my Master’s Thesis (A 3D Mapping Payload for UAV Applications) in 2013. Then, I started working full-time with it around 2016 and never stopped. My main focus was always on applications using drones and mobile robots, so covering mobile robotics was a natural fit for the whole project series.

Why did I switch from calling it a course to calling it a project? Manning LiveProjects are not traditional online courses. Instead, the students get the materials they need to go through by themselves. Then they complete the assignments themselves, with occasional help from extra resources, mentors or fellow students.

Project Series

In the project series, you will be joining RoboPrototypes, a company focusing on developing custom robotics solutions for their customers. Once you complete your first project, you are on board with the company, and you have to deliver Dribot, a mobile robot prototype that will carry drinks in the customer’s food courts in the mall near you.

A simplified 3D render of a Dribot differential-drive robot

Your quirky boss Michael (any potential resemblance to characters from a certain TV show about people working in an office is purely coincidental), who at one point is careless enough to hit your co-worker’s shin really hard with a robot, will be closely monitoring your progress.

There are two highlights of my liveProject series that I think can make it interesting for students:

The author’s insights build on my expertise working in the past four years as a consultant, dealing with a platitude of robotic platforms (more on this in the next section)
Mentorship - if any student gets stuck in a project, they can reach out on the chat to get help from me

Series Structure

I love the modularity that comes with ROS. Thanks to it, I was able to structure the course into the following sections that make logical sense:

ROS2 - Getting Started - where we follow the official tutorials and create some simple launchfiles and nodes
ROS2 - Simulate a Robot - where we create a robot description and simulate a differential-drive robot in Gazebo Classic
ROS2 - Sensors and Sensor Fusion - where we add a Camera, LiDAR, and IMU and implement robot_localization for sensor fusion
ROS2 - Simultaneous Localization and Mapping - where we use Slam Toolbox for, well, SLAM
ROS2 - Navigation - where we use Nav2 for navigation in the AWS RoboMaker Small House World

With the above projects, each consisting of three to five milestones, the work should be divided into manageable tasks. My idea at the time was that each of the milestones should take at most one evening to implement, and I hope I achieved this goal.

Resources

A project is as good as the resources contained within

Me, right about now

Since the idea behind the LiveProjects is to provide resources to students so that they can complete the project themselves, it’s essential to find quality resources. Fortunately, right before I started working on the series, Andreas Bihlmaier began working on his book Robotics for Software Engineers. Andreas does a fantastic job explaining robotics concepts, I mean, take a look at the image below:

Sense. Plan, Act in various robotic systems. Source: Robotics for Software Engineers, Chapter 1 by Andreas Bihlmaier

Since both of our projects are with Manning, I can link to Robotics for Software Engineers throughout my liveProject series. Additionally, students get a couple of months of access to the book when they purchase a liveProject

On rare occasions, I could not find materials that would explain some concepts easily enough. In such cases, I wrote the introductions myself or created a dedicated blog post in one case.

Author Insights

As I mentioned earlier, the author’s insights are my favourite bits since they allow me to share some experiences from the field. Here is a sample from one of the milestones in the Sensors and Sensor Fusion liveProject:

Almost every client I have had that was working with mobile robots had some issue with an IMU. These insights will be quite long, but building intuition about IMUs will help you build quality robots. I will have some insight to offer in the next milestone as well.

Transformation frames

If you’ve gone through the previous project (Simulate a Robot), you will remember we discussed coordinate frames. You might remember that for a typical robot platform, you would expect to have an x-axis pointing forward, y-axis pointing left, and z-axis pointing up.

The thing with sensors is that they will always report data in their local coordinate frame, so you might find an IMU that, when in a default position its z-axis is pointing down. (Let’s say that the x-axis is still pointing forward; this means that the y-axis points to the right).

The first thing we need to take care of is to make sure that our imu_link has the same orientation as the orientation of the IMU coordinate frame (in this case rolled 180 degrees around our robot’s x-axis).

This situation will get tricky when you start analyzing raw data. Since your IMU data will be referenced in the IMU frame, this means that if your robot rotates counter-clockwise it will report a positive angular velocity around the z-axis, but your IMU will report a negative velocity! This is totally fine as long as the imu_link correctly describes the orientation of the IMU coordinate system.

Any system that takes in this velocity will need to look up the coordinate transform between the base_link and the imu_link.

Vibration

A good rule of thumb is to isolate the vibrations physically. Ideally, you should put a rubber pad or thick double sided tape between the IMU and your frame to dampen the vibrations between the frame and the IMU.

Even though you could filter the IMU data in software, I would recommend always trying to solve the problem in the physical domain first whenever dealing with sensors or actuators and only then start looking into the issue through software.

More insights on IMUs are coming soon in the following milestones!

Statistics

Here are some statistics on the project series that you might find interesting:

Months from project start to release: 7
Working hours to deliver liveProject: 177.5 hours
Working hours spent on project mentorship so far: 9 hours
Total number of text for the whole project series: 34,019 words OR 234,120 characters OR 98 A4 pages (font size 10)
Total numbers of commits in my project repository: 146
Number of certificates granted so far: 2
Number of students that signed for the free project so far: 258
Number of students that signed up for the follow-up projects: 97

The Difficulties

One thing I did not figure out in this project series, and I would be super grateful if anyone has any ideas for a clean implementation, is handling a consistent code base along multiple projects. We are building up on the same code base from the second project until the end. Each of the milestones adds a bit of new code or configuration, and when I started, I was copying the source code between milestones and appropriately providing partial or full solutions for each of them.

The above means that if there is a bug in the first milestone of the second project, I would need to manually copy the fix over to all the other milestones (I did it twice already). Perhaps a cleaner solution would be to use a branch per project milestone, but this way, I would need to apply any fixes to up to 16 branches, not really reducing the complexity of this operation.

If you have any thoughts on this, please let me know. If you would like to create a project like this, I suggest you have a good think about this issue before you start.

Outro

Dribot navigating through waypoints in a simulated apartment

This project was quite a ride. The main challenge for me was consulting close to full-time and then also working on this series. With the help of Manning Staff, outstanding Reviewers and Implementers, I delivered what I think is a solid ROS2 project series for beginners who would like to learn how to use ROS2 for mobile robots.

Updating the whole series to work with Gazebo Fortress and ROS2 Humble is still on my plate. Hopefully, I will be able to deliver this summer.

That is all that I have for this update. Happy roboting!

I Created a Project-based ROS2 Course was originally published by Mateusz Sadowski at msadowski blog on July 05, 2022.

IMUs and LiDARs - Not Uncommon Pitfalls

2022-02-24T00:00:00-00:00

While preparing my ROS2 course on mobile robots I was not able to find quality information about basic sensors that can be used for mobile robots and the most important considerations on using them. In this short blog post, I’ll discuss some of the issues I have come across while using IMUs and LiDARs in tens of robotics projects.

IMUs

If you are working in mobile robotics it’s pretty much given that you will use an IMU on your robot. Over the years I’ve worked with multiple sensors such as:

MPU6050
Bosch BNO055
PhidgetSpatial Precision
Xsens MTi-630-DK - last used for my blog post on 3D Mapping

When choosing a sensor, your number one consideration will be most likely price, if you are on the market for some sensors the IMUs above will most likely introduce you to the whole range of prices. Please don’t consider the list above as advice, as everything depends on the fine details of your project. Most likely you will want to take a look at such features as:

Update rate - for wheeled mobile robots ~50Hz is enough, for UAVs 100-500Hz should do depending on aircraft type
Whether the attitude fusion happens onboard the sensor
Available interfaces - USB is probably the easiest if you are thinking of using ROS
Resolution, maximum readings, gyro drift etc.
Calibration requirements - more on this in the next section

IMU gotchas

In my experience so far, IMUs are the sensors causing the most trouble when integrating them on mobile robots. Let’s go through some common pitfalls one by one:

1. Magnetic interferences

In my first job, we were integrating a drone autopilot with an onboard IMU on a helicopter platform. After many days of trial and error, we were not able to tune the heading controller that would make the helicopter keep a fixed heading while in the air. Only by accident, we’ve noticed lots of noise in the heading component in the IMU data and then when we were landing the aircraft we’ve noticed that, actually, these magnetometer errors have a period and it’s tied to the rotation speed of the main rotor! We have not seen this behaviour in the smaller model, which lead us to the discovery that in the model we had used ferrous metal rods in the blades! That would explain the noise.

Rough dimensions of the helicopter in question (not the exact model). Source: align.com.tw

HINT: If you are using a magnetometer in your robot - always move it as far away as possible from sources of the magnetic field. The most common source would be moving ferrous metals, actuators (magnets!) and high-current wires.

2. Vibration

Your robot most likely vibrates. The amplitude of the vibrations will depend on materials used, type of terrain and the drive train (among many other factors). In drones, we had a rule that when you plot all acceleration axes, and X or Y axis measurements are touching the Z-axis measurements during hover then your vibration levels are too high.

Example of vibrations that are definitely too high. Source: docs.px4.io

Transferring these rules onto mobile robotics:

HINT: If your robot drives on a flat planar surface and you see acceleration readings overlap between X/Y and Z axes then you definitely need to dampen the vibrations.

To dampen these kinds of vibrations you could use rubber offsets or thick double-sided tape.

3. Axes definition

This point is highly specific to software, but many robotic systems I’ve been working on had issues with wrongly defined IMU axes at one point or another.

HINT: Always double-check the IMU axes defined in your software vs. the axes of your physical device.

4. Calibration

Some IMUs will require you to perform calibration (usually rotating the sensor in some way in all-axes), however, some IMUs will not store this information (looking at you BNO055). Once your IMU is mounted on a target system, you might not be able to rotate it along all axes, so you need to make sure this will not cause you some issues.

3D IMU calibration after you've fitted it on the car. Source: YouTube (Top Gear)

LiDARs

If you are reading this article from top to bottom (why wouldn’t you?), then here starts the fun! With LiDARs we are entering a space of ‘smart robotics’, where our robots can start reasoning around their environments and can know where they are.

Without going into too much detail (Wikipedia has got you covered for this), with LiDARs we are talking about a light pulse is emitted and we calculate the time it takes for it to come back (in some cases we might look at the phase shift of light too).

When we talk about LiDARs on the market we will find the following types:

Single point distance sensors (e.g. Terabee TeraRanger Evo, Lightware SF11/C) - provide single point measurements
2-D scanners (e.g. RPLidar A1, Hokuyo UTM-30LX) - provide single plane measurements
3-D scanners (e.g. Ouster, Velodyne Puck) - provide measurements in multiple planes
Scanners with non-repeating patterns (e.g. Livox Mid-40) - provide measurements as a non-repeating pattern

The choice of the sensor should depend on your application, interfaces you want to support, the processing power available etc.

LiDAR gotchas

Before I worked with LiDARs I didn’t know that light can be so annoying! Here are some things you might (but hopefully won’t) come across.

1. Eye fatigue

Now, I don’t have any evidence for this, but on numerous occasions, I have experienced eye fatigue while working with LiDARs. It feels like your eyes are a bit numb, my colleague describe the feeling as ‘pain, but without the pain component’. Technically, most LiDARs you will come across will be eye-safe, but I would still recommend not keeping them at your eye level. If anyone knows any research about long term issues that could result from these, please feel free to let me know.

Me, the first time setting up the robot for half a day and keeping the LiDAR at my eye level

2. ‘Flip effect’

This is one of the issues that you will rarely come across but if you do it will be a huge pain in the ass to rectify. I’m actually not aware of anything you can reliably do to filter this kind of data in all cases. What happens here, is that the sensor reports a way smaller distance than it should, as shown in the picture below:

The 'flip effect' illustrated

The real distance of the sensor is red + green (expected distance), but the sensor report just the green distance. The only way to fix this problem that I know of is to update sensor firmware.

3. Field of view shenanigans

Your LiDAR emitter will have a certain field of view (most likely a circular one, about 1-3 degrees). Looking at the picture below, there is a sensor with a field of view, and an object that is only partially in view. What distance would you expect the sensor to provide?

Sensor and it's field of view

The answer is: it depends on your sensor! Some will provide the first return (Y), some will provide both Y and X (that’s how you can map tree canopies with LiDAR) and some will average all distances within the field of view providing the result Y < Result < X.

Whether any of these will be a problem for you depends entirely on your application.

4. Maximum distance vs. environment

The datasheets of LiDARs will usually specify the maximum distance, but you will not necessarily see the same maximum distance in your application. Usually, the specs provided are for targets with some defined reflectivity. If you direct your sensor at a black target (low reflectivity), you probably won’t be able to reach the maximum distance shown in the datasheet.

Another issue that will hamper your measurements are outside conditions. The chart below will show you which wavelengths are overly present in solar radiation.

Solar radiation spectrum. Source: Wikimedia Commons

Most likely, the sensor of your choice will fall into one of the valleys in the chart. If the manufacturer of your sensor is not providing the wavelength of the sensor. consider it to be a red flag. Combining everything we’ve covered in these subsections, you might learn that having a sensor operate in full sun, looking at tall grass might yield the maximum range that will be suboptimal for your application.

Outro

This about sums it up, when it comes to some pitfalls I’ve run into when working with LiDARs and IMUs. Robotics being robotics there are hundreds of other things that can go wrong in your setup, hopefully with some patience and good detective work you’ll be able to get to the bottom of issues you come across in your setups.

Now, let’s make some robots!

Turtlebot wearing a Weekly Robotics T-shirt

IMUs and LiDARs - Not Uncommon Pitfalls was originally published by Mateusz Sadowski at msadowski blog on February 24, 2022.

Remote Robotics Consulting - 3.5 Years In

2021-10-31T00:00:00-00:00

This is the third blog post in the series of posts summing up my experience working as a Robotics Consultant. My recent realisation is that I’ve been consulting for half my career now, so hopefully some of the things I’ve learnt along the way will help someone else out there.

Previous Blog Posts

If this update is your entry point to my reflections on working as a remote Robotics Consultant, then you might want to start with my previous blog posts: 1 Year In, 2 Years In. I’ll do my best not to talk about the same things, but instead focus on the things that have either changed or that I’ve developed new thoughts on.

High Level Business Update

Let’s start with the client maps, as I’ve had 10 new clients since the last update:

Clients I've worked with since starting

Clients I've worked with since starting - EU edition

When it comes to clients, there have been some significant changes to my approach since the last update. I have two different rates for clients now. Simply put, for short-term engagement work, I charge more than for long-term clients. I’ve found that this approach has helped a lot with my work-life balance and that the premium for short-term consultations adds some motivation to fit them outside my standard working hours.

In the previous update, I mentioned that I’ve started evaluating EU funding proposals to make sure your tax money is spent well. I’m still doing that and so far I’ve been involved in five evaluations. Even though these evaluations don’t pay nearly as good as consulting, they are a very good business exercise. On top of this, ensuring that tax money is spent on quality projects that can really elevate companies makes me feel good!

Pandemic and Health

I’ve been lucky that the COVID-19 pandemic did not affect my projects and so I’ve maintained a steady stream of work since I started consulting. I also didn’t catch COVID, which was a plus! The way COVID did affect me, however, was that I was days away from burning out. This is because I took on too many projects than I could handle, and got bored out of my mind with all the social restrictions and locked gyms.

Then, once everything started opening up again, I continued overshooting:

I kept the same number of projects (often worked on weekends)
I started going to the gym
I continued running

I got to a point where I wanted to make up all the time away from the gym and so I was going to the gym three times a week, and running in the days between. After about 2-3 weeks of this, my body seemed to have had enough and I lost all energy for about three weeks; I had issues with getting enough sleep and as a result, I was not really capable of doing any exercise.

As I’m writing this post, I think I’ve found a balance with going to the gym three times a week and running at the gym for a short period of time. The risk of burnout also seemed to have gone away with the realisations described in the next section.

Working with Clients

I’ve found that a switch to remote working for many offices due to the pandemic actually made me more ingrained within the teams I have worked with. Usually, the teams I work with are within the same building, but with everyone switching to remote working, we were all constrained to the same means of communications.

Another thought I have about remote work is that in one of my previous posts I’ve mentioned that it’s hard to work remotely with hardware. I still hold that opinion, but I’ve found that with experience, it’s less and less time consuming, as long as the team ‘on the ground’ can test my updates and provide me with feedback.

When I was writing my previous updates, around 90-95% of my projects were from Upwork. Sometime at the beginning of 2021, there was a big reversal, and nowadays more people reach out to me directly, than through Upwork. This works quite well most of the time, as I don’t have to pay 20% platform fees for short projects. What I miss though is the payment security. So far I’ve been sending out invoices after the consultations, even for new clients, but as some customers have been late with their payments, I’ll probably need to look into being paid upfront for the first few engagements. The risk is manageable for short-term consultations, but investing one month into the work, and then stressing out about the customer being late on payment is something I would not recommend.

Some of the robots I contributed to in the past year and a bit

When it comes to consulting projects in the past 1.5 years, I focused on the following:

A modular mobile robot with Ross Robotics.
An unmanned boat for automatic scanning bodies of water with Simple Unmanned < a nice dive into bathymetry (pun intended).
An industrial robotics project that I won’t share any details on at this time.
Multiple 1 hour-long consultations on SLAM, LiDARs, mobile robots.
The EU evaluation projects I mentioned previously.

Time Management

In my previous ‘2 Years In’ post, I mentioned the negative aspects of context switching, how I was trying to avoid it and how I would do it, at most, 2-3 times a day. In the past three months or so, I have taken it to yet another level and started booking ‘customer days’. This fully eliminates context switching, and if I’ve agreed with a customer to be available on Mondays, Wednesday and Thursdays, then they are assured I’ll be reachable. If I have any extra consultations, they come after-hours.

There is one more realisation I had about time this year: I can’t work more than six hours a day on tasks requiring a high level of focus. This will vary slightly depending on the task, but I’ve noticed that accepting this has resulted in the following for me:

Feeling way better with my work-life balance.
Taking less projects -> feeling less stressed (consultants who went through the conjunction of deliverables for 3 projects at the same time will know what I’m talking about).
Producing higher quality results and thus being more cost efficient to clients.

Weekly Robotics

Weekly Robotics Logo

Weekly Robotics, the newsletter that I’ve been maintaining since August 2018, is still my favourite project. Thinking of it, it’s crazy that I haven’t missed a single issue since I started this project. So far this year I’ve clocked 170 hours working on this project, and right now we are sitting at 2,700 e-mail subscribers.

Subscription growth for Weekly Robotics

The biggest introduction to the newsletter this year was the meetups. In the first season, we had 12 presenters on all kinds of topics related to our industry. We have restarted the series this fall and I’m really looking forward to keep on learning! If you would like to join any of our meetups, then you can either subscribe to the newsletter, or follow us on Eventbrite.

The only recurring funding I get for the newsletter is the Patreon, where we have 21 patrons. Occasionally, I post referral links to the Humble Books Bundle (just like this one) related to technology. If a reader decides to purchase the bundle, they can choose to give a portion of the price to the newsletter. So far, most of the sponsored posts were me bartering the space in the newsletter for discounts/media sponsorship/robotic hardware.

I don’t think the newsletter will ever come close to supplying as much profit as consulting does, given that I spend 3-6 hours a week on it, but I think the newsletter is a great investment for the following reasons:

I have a good motivation to stay up-to-date on the industry.
I love that I have a chance to discuss thoughts and ideas with industry experts on the WR Slack; having some sense of community around the project feels good.
It helps a lot with discussions with potential clients, as it shows I’m consistent.

Life Updates

I’m not sure if anyone cares, but in September 2020, I moved to Prague, rented a workshop and proceeded to take things to the next level. City-wise, I love it! There are so many activities to choose from, nice parks and good beer, but I have to say, I miss the French mountains quite a bit.

The office

Panorama of the office

I still have a long way to go with the workshop, but having so much space (45 square meters to be exact) for activities is great. I also discovered that I’m much more productive in the office than at home. Having a clear separation between work and home life seriously boosts my productivity. Another plus is that it’s much easier to keep the rooms clean without having so many tools lying around.

Future Plans

I plan to keep on the current track for the foreseeable future. I will definitely keep acting on my ROS2 FOMO, as I quite enjoy it so far - more news on that in about six months!

Even though I’ve learnt to say no to the robotics projects, I still can’t seem to say no to more exotic opportunities. This usually ends up as follows:

Of course I will be a technical reviewer for your robotics book.

You want me to develop a ROS course? Say no more!

I’m supposed to be an influencer to your brand on LinkedIn? Let’s do it!

Yes, I will join your Board of Directors (this hasn’t happened yet, but I have a strange feeling this will be my answer).

I think these are all the thoughts that I have to share this time. Thanks for reading and keep on roboting!

Remote Robotics Consulting - 3.5 Years In was originally published by Mateusz Sadowski at msadowski blog on October 31, 2021.

Making a 3D mapping prototype with ROS

2021-06-28T00:00:00-00:00

In my previous post I’ve shared some of my experience working with Septentrio Mosaic X-5 dev kits. After seeing some good results with that hardware I’ve decided to try to use them with some other hardware to realise a prototype that I had in mind for quite some years. In this post I’m going to describe what I did to get a first prototype working and share some tips for anyone who would like to replicate them.

The idea

I’ve been interested in developing a proof of concept system for mapping (either using SLAM or other methods) for at least 7 years now (it might be slightly related to the fact that I was developing something similar for my master thesis). Since I’ve had a chance to work with RTK GNSS units (firstly with ArduSimple and then with Septentrio’s solutions) the logical next step was to start integrating more sensors. Before we get into the nitty-gritty details and the final demo I run let’s take a look at the first prototype of the idea:

Indoor scanning

The design

Here is the list of hardware that I had access to for this project:

2x Septentrio Mosaic X-5 dev kit - precise GNSS modules in RTK setup
Xsens MTi-630-DK - AHRS (a precise magnetometer was a must-have for the prototype)
Livox Mid-40 LiDAR
Two Mikrotik routers

Sensors

Livox Mid-40

Livox Mid-40 LiDAR

I’ve received Mid-40 from Livox over 2 years ago and described my initial thoughts in this blog post. Having a neat scanning pattern was an interesting feature for the prototype. With a single plane LiDAR, I would have to move the prototype quite a bit to gather data, with Mid-40 I could instantly tell if the data was good because I would immediately see the features in the scans.

Xsens MTi-630-DK

MTi-630-DK mounted on the prototype frame

Xsens was very kind to lend me an evaluation unit of MTi-630-DK for my demonstrator, even though the initial phase of the project took me way longer than anticipated. The MTi-630 is an Attitude and Heading Reference System (AHRS) as it runs on-board sensor fusion algorithms that estimate absolute 3D orientation, in this blog post I’m using IMU and AHRS interchangeably. The main feature I was after in the IMU/AHRS was as precise globally referenced heading as I could get. The better the AHRS the less angular drift I would expect to see in my data.

In the video I’ve shared in the previous section you can see me testing the first prototype in a room running just the MTi and the Mid-40. I used double sided-tape to attach the unit to the top of the LiDAR, created a simple URDF file with correct offset between frames and voila! By slowly rotating the LiDAR in place I was able to scan the room. Let’s take a look at it again:

Indoor scanning

Do you see an issue with the 3D map being created? As the LiDAR is turning left you can notice that the new points appear shifted with regard to the points that were initially captured.

Bad map!

In my hastily prepared prototype, I assumed that the Mid-40 origin is in the centre of the LiDAR, while according to the docs the origin of the LiDAR is flushed with the front face of the unit. Oh well, it’ll be fixed in the next iteration that will include the next sensor.

And if you are thinking now “hey, Mat, won’t placing the sensor so close to some moving parts, that are undoubtedly magnetic and surely inside the LiDAR, affect the sensor readings?” then you are right. Even though I couldn’t see it when looking at RVIZ data, much later I noticed that turning on LiDAR affected the magnetometer reading that was fused by the AHRS for the orientation estimation.

Septentrio Mosaic X-5

Septentrio Mosaic X-5 Dev Kit

I’ve used Septentrio dev kits in exactly the same way that I’ve described in the previous blog post. Just as a quick recap: one unit was functioning as a base station, sending corrections to the rover unit that I’ve mounted on the prototype. All exchanging information through WiFi.

Having very precise location information is the core problem to solve for this prototype. Another approach that could be used would be to try to run SLAM, but this would require more processing power and most likely a LiDAR with a more repeatable pattern and higher field of view.

Putting it all together

Hardware

Here is how I connected everything up when it comes to hardware:

Simplified hardware architecture of the system

I’ve mounted all the components on a frame built from 2020 V-slot profiles, making sure I have enough room for anything I might want to add in the future:

First prototype (before moving IMU to the bottom frame)

The LiDAR and the Router were both powered off a single 3S LiPO battery, while the GNSS unit and the IMU was powered from my laptop’s USB ports.

There are two issues with this setup:

I planned to use my laptop for processing and could not be bothered setting up a smaller processing unit
The unit weighs too much to be comfortably carried for long periods, even if a 2nd person carries the laptop

The best thing about this setup was the flexibility for mounting things. I had enough space to keep adding any hardware I might have needed or wanted to try out. After all, my only goal in these experiments was to prove the concept.

Software

For my prototype, I’ve used ROS Melodic, since that’s the version I’m still mostly using, and I had it on my machine. Something I love ROS for is the existing packages and libraries. What I think is interesting, is that for the prototype I didn’t need to write any software, I was just able to put together a bunch of existing packages by having the right configuration (perhaps having a good idea of what I’m doing helped too).

ROS nodes I've run for a proof of concept

To make testing easier I’ve created a xacro file that defined all the dimensions and components, together with their position. Bein able to visualise the setup made testing easier as I could double check the state estimation matches reality:

Sensor placement in urdf/xacro

This data on where sensors are with respect to one another is used throughout the system. Not only it allows to accurately represent the PointCloud in RVIZ but it also is used by EKF nodes for checking the relative positions between elements.

The localization is a classic ROS ekf stack described in detail on these pages. Running this EKF allows us to get an estimated position of the robot in a fixed coordinate frame (map or odom). Setting the Fixed frame to map in RVIZ then allows us to visualised all the LiDAR points as you can see here:

An OK mapping run

This is one of the 2 mapping runs I’ve run with this hardware, with a hand-tuned heading offset. This result is exactly what I need to prove the concept. If you know Pareto principle I firmly believe that in 20% of the time, I’ve shown 80% of the system functionality. The remaining 80% time required is a solid grind, with lots of tuning and field trips to create something viable.

Lessons Learned

I’ve learned a tonne in this project, so naturally, there is some brain dump:

V-Slot 2020 profiles are amazing, if you work with them I highly recommend 3D printing or sourcing some caps that you can place on the profile ends. They are quite sharp, without caps you don’t want them anywhere near surfaces you care about
Start work with some computer that will run everything and have a small screen, running everything on the laptop, and carrying a heavy prototype is not a great idea
In my experience so far, every trial takes me between 3-5 hours, mostly because I’m in the city, and quiet places with good views of the sky are hard to come by. If you are using ROS then record bagfiles and use them for tuning the system (especially localization and IMU integration)
Use checklists - on one occasion I had all the cables I needed by pure luck. I highly recommend using a checklist for field packing and performing experiments. Ensuring your GNSS unit is connected to the router when testing will surely save you some debugging time as well!

Further thoughts

For a project like this timing is everything. Both the LiDAR and the IMU that I’ve used support sync input, while Septentrio’s Mosaic can easily be configured to provide sync output. Instead of using EKF as I did, perhaps the data could be used directly with very little sensor fusion. This would probably be the approach I would have taken if I was developing a handheld module.

Instead, I’m way more likely to put this kind of hardware on a mobile robot of some kind, making sensor fusion a much more likely requirement as the precise position and orientation would be used for other aspects than mapping.

Next steps?

Right now I don’t have any plans to pursue this project further as it is. I’m glad that I was able to prove the concept but I don’t have a use case for such a device. I’ll use these learnings in future robot integration projects. If someone wants to build off the work I’ve described here for an open-source project then I’ll be more than happy to help out, but for now, it’s time to focus on other ideas that I have in the pipeline!

Making a 3D mapping prototype with ROS was originally published by Mateusz Sadowski at msadowski blog on June 28, 2021.

Plug&Play RTK with Septentrio mosaic-X5 Dev Kit

2021-03-31T00:00:00-00:00

Quite a while ago I’ve received two units of mosaic development kits from Septentrio. This blog posts sums up my experience with it and provides some insights into integrating these with ROS.

First impressions

The first thing that you notice when unpacking the module is the size of the carrier board at 10x16 cm it’s quite large, however, the core module is only 7x5 cm.

I don’t think I’ve ever used a positioning system that would be that simple to set up. This video that is less than 2 minutes explains the basics on how to connect to the module. The gist of it is you connect the receiver antenna, attach a USB cable and navigate to the local webserver, hosted by the module in your web browser (192.168.3.1) and poof the interface is there:

Septentrio Web Interface

Need to set up the module to output NMEA sentences on a USB port? Simple! Go to NMEA/SBF, select connection type (USB), desired serial port, message types (e.g. GGA, RMC) and interval you are interested in and you are good to go. The dev USB port on Septentrio (the micro-USB port on the carrier board) creates two serial ports that you can use on your target machine. Being able to configure UDP or TCP streams is also an interesting option.

The base station-rover setup is very easy as well - the instructional video has less than 3 minutes, once you know what you are doing this setup likely takes way less than a minute.

What is also worth noting is the number of carriers that are supported. In my test the base station was receiving position from 4 systems: GPS, GLONASS, Galileo, BeiDou:

Ground station status page

Testing

Hardware setup

A mount I 3D printed to attach mosaic dev kit to tripod and hold antenna

To run some integration tests with ROS I’ve decided to make two tripods, each carrying a Septentrio mosaic-X5 kit, antenna, battery and a wireless router. One of the tripods is used as a base station, while the second one is used as a rover (I’ll carry this one in hand when testing). Additionally, the rover has an IMU with a magnetometer attached next to the antenna.

Base setup

As I mentioned in the previous section I’ve set up an NMEA USB stream for my tests. To do this I created a new udev rule for the modules (the full workflow being similar to the one described in my static USB tutorial):

KERNEL=="ttyACM*", ATTRS{idVendor}=="152a", MODE="0666"

I’ve set the two MikroTik routers to PTP Bridge mode, making sure the base station is outputting correction on TCP port and that the rover module is subscribing to them.

ROS setup

ROS setup is quite classic for mobile robot setup using navsat_transform_node, robot_localization, imu driver and navsat_nmea_driver. Let’s go over the most interesting bits of my test setup.

Navsat driver

I’ve decided to use the nmea_navsat_driver for reading NMEA data from Septentrio through a serial port. The reason I opted for this is the simplicity - I could set it all up and be testing in no time. Or so I thought until I’ve discovered that nmea_navsat_driver has this thing that will prepend the frame_id you provide it with /. This in turn breaks the latest version of robot_localization. This seems to have been fixed in Noetic but is a wontfix for Melodic.

Localization

Since for quite a while I’ve wanted to create a handheld device for some data capture (hopefully more info coming soon) I’ve decided on the following assumptions:

Single EKF - for any mobile robot setup you would most likely use a dual EKF setup but since I don’t have any sensor providing continuous measurements in planar axes a single EKF instance should be sufficient
IMU fusion for orientation and heading - could be useful in the future when I start pointing my setup at things
WGS84 -> fixed frame transform - I’ve set up robot_localization to publish this transform so that mapviz would display my ‘robot’ in the global frame

Test results

For this blog post I’ve performed two experiments:

Static test - both rover and the base are placed in a static location for 15 minutes, then for around 10 minutes we run the localization stack and observe changes to the local position
Dynamic test - trying to trace a pattern drawn on the ground

If you are a robotics engineer and not getting enough sun exposure then I highly recommend testing GNSS modules, this setup allows for some proper vitamin D boost:

I'm really glad I bought camping equipment at one point of my life

Static test

At first, I was thinking of setting up a 10x10cm grid in mapviz and test if the base frame in the local coordinate system ever moves outside the grid cell during the test. Boy, was I pessimistic…

The test I actually performed was launching the robot_localization, causing the local frame to be at the initial position of the rover receiver, and letting it run for around 10 minutes without touching any tripods. Here is the X and Y position (thanks PlotJuggler!):

Graph with X and Y position change over time

As you can see in the graph, throughout the test the position didn’t significantly move. The lowest measurement on the above graph is -0.010895 m (that’s < 1.1 cm). The ground I’ve set up on wasn’t flat and it was quite windy. I have no idea if these factors could cause the module to shift slightly but the result is amazing as it is, working on wheeled-mobile robots I would not need higher precision anyway.

Covariance recorded during the static test

The covariance output from nmea_navsat_driver looks a bit higher than I would expect from a unit with an RTK fix. I will need to check how it’s calculated as the high values of covariance could be an issue when fusing in wheel odometry in later stages of the integration.

What is interesting in the static test is the precision of height data that is not far away from what’ve seen for XY-axes:

Relative height captured throughout the test

Dynamic tests

The dynamic tests I performed suffer from one issue: I don’t have any reference information to compare the measurements against. For having the absolute idea about the quality of measuerements some proper geodetic methods should be used (for example a total station).

For this test, I traced a roughly rectangular path and walked around it carrying the rover unit. The measurements won’t be very consistent as I was holding the whole rover setup in hand while walking the marked path. Here is the GNSS odometry position:

GNSS pose in local frame

The data looks very good, I believe once covariance is addressed this setup should pretty much work out of the box on any mobile robot as the global EKF providing map->odom transform.

Closing thoughts

When I mentioned Plug&Play in the title of this blog post I wasn’t kidding. I like how Septentrio made it easy to work with these modules. Everything I needed to set-up was explained in the videos, with these tutorials rarely exceeding 3 minutes. The quality of support and documentation that you get from Septentrio is also top-notch.

At the same time there are some features that I will be looking forward to testing in the future:

Using SECORX positioning service and/or NTRIP service integration for corrections
Enabling PPS and NTP server on the mosaic’s side
Switching from NMEA output to SBF (Septentrio Binary Format) and testing ROSaic driver or mosaic_gnss_driver
Upgrading the firmware on Septentrio and running some more test on increased update rate (so far I’ve been outputting position at 10 Hz)
Eventually testing mosaic-H with dual antenna setup for receiving heading information

Now, what would be a really, really interesting test? Taking this setup, adding a quality IMU and a LiDAR unit to it, creating a very precise 3D mapping module.

Plug&Play RTK with Septentrio mosaic-X5 Dev Kit was originally published by Mateusz Sadowski at msadowski blog on March 31, 2021.

Thoughts on 2 Years as a Remote Robotics Consultant

2020-08-26T00:00:00-00:00

It’s that time of year again when I’m thinking back on the journey I set up for myself by deciding to go into Robotics Consulting. This article is a follow up to the blog post I wrote last year. This year, I’ve decided to try and cover the most important aspects of what I’m doing, and how it has been working out for me.

The work I had

I’m very grateful for the kind of work that I do - it feels amazing doing what I love and getting paid for it. I would never change my work for anything else. Every single one of my clients has an interesting problem to solve, and I’m yet to get my first negative experience working with someone. Looking back on all my clients, I’ve worked with many interesting people all around the world, mostly helping them to develop software for mobile robots and drones.

A world map of the 22 clients I've worked with over the past 2 years

My European clients

The duration of the projects I take on are between half a day and 2.5 years long. Everything depends on the client and what they require. Most of my projects kick off with a consultation - helping the client to solve a particular problem by sharing my expertise and experience. After this initial phase, some of the clients decide to hire me as a contractor, helping their technical team solve the challenges they face, while others are happy to carry on working by themselves with the advice received.

As a tangent to the consultations, I also help evaluate robotics funding proposals, as a reviewer in EU technological funding projects ESMERA and TRINITY, making sure your tax euros are spent on truly cutting-edge projects.

Putting myself in my clients’ shoes

Every project I take on, I treat as my own. This results in me trying to solve issues before they happen, and sometimes even pushing back on client’s decisions - “How about we don’t remove this safety feature?”. Another side effect of treating the project as my own is that I put the client first, before my own business interest, meaning I might earn less for the project overall.

As an example, I was hired to test the idea a client had for a robotic project. The idea turned out to be borderline feasible, with currently available technology, but potentially requiring lots of R&D work. If I sold the project as “no problem, it’s totally doable”, I would definitely get more work supporting the R&D work required. By putting the project and client’s interests first, however, we finished the project after the feasibility study, having realised the extent of R&D work needed.

Since terminating such a project before going to an R&D phase is something I would do, I’m content and the client is most likely even more so, since they don’t have to spend lots of money, not understanding the R&D effort required. Instead, they can decide whether to pursue the idea further. If the client does decide to take a risk, they know that I’ll be there, ready to treat their project as my own, and providing honest advice, even if it means I get less work from it.

The hardware

Working with hardware is one of the most enjoyable parts of my job. This year, I’ve been focusing a lot on RTK solutions (I have one blog post in the works on a neat RTK setup). Also this year, I finally got my hands on a multi-plane LiDAR, something I’ve been very keen on ever since Velodyne revealed their first multi-plane units.

Some of the hardware I've worked with since I started working as a Remote Robotics Consultant

The Work I Did Not Win

The dream project

At one point, I was approached by a client who wanted to create a certain type of robot that I’ve always dreamt of working with. The task would require me to liaise with manufacturers and developers to get the robot ready, and enable support for autonomous navigation in various terrain types. In short, a hugely challenging project that would immensely help me grow, whilst being super fun at the same time. When discussing the project in more detail, I noticed that the budget for this client was not an issue - any platform for any price was OK, as long as it met the requirements. However, when asking follow up questions, it became clear that the project target was to equip robots with weapons.

The kind of project I’m not keen to work on. Source: Bosstown Dynamics (Corridor Digital)

When I was applying to University, I promised myself I would not work on weapons, even if these projects have a virtually unlimited budget. I figured that even if I was to fold my consultancy, I would rather do that than compromise on my values.

The bargain

In the two years working as a Robotics Consultant, I’ve noticed a pattern: the more a potential client bargains before starting the job, the more bargaining and complaining will follow, resulting in an unpleasant experience for everyone involved. In the same bucket as bargaining clients are the UpWork projects that want you to create a SLAM library from scratch for a fixed price of $50. These days, I tend to reject these types of clients and contracts straight away, saving everyone’s time.

Watch out for the legal stuff

If I’m receiving 25 pages of legal documents to review before engaging with a customer, it’s a potential red flag. I don’t mind reading legal documents, but I would rather not involve a lawyer before even starting a project. For me, a huge red flag is when a potential client adds a very vaguely described 3-year non-compete clause, covering a whole industry. I find such terms too restrictive and would never take a project on like this, unless the project also paid for a 3 year holiday after wrapping everything up!

The lack of expertise

As much as I would love to know everything about Robotics, it’s not likely to happen anytime soon. I will never take on projects that fall outside of my expertise (for example, algorithms for soft robotics), unless the client is persistent and understands an R&D process. My go-to advice in such cases is that it’s not feasible to pay my high rates for me to catch up on the topic. In a hypothetical situation, if the project falls outside of my expertise, but it’s in an area I’m planning to pursue, I’d offer a very low rate for the project, highlighting that I will need to do some catching up.

Remote Robotics

People often ask me how I work on robotics projects remotely (I’ve been doing this since day 1 of my consultancy). When I’m doing some high level work, it’s usually not an issue. It only becomes slightly more problematic with hardware-oriented projects. The options I’ve tested so far are:

For ROS systems, working with bag files, and in the case of drones, working with log files
Clients shipping the hardware to me
Travelling to the client’s location
Remote control

Me and my partner's office in the time of the COVID-19 pandemic

ROS is absolutely the best thing that happened to people like me - I can easily review things online and even develop software with just bagfiles. However, it’s not always feasible to work with bags, especially if you need to do some work related to sensors and actuators. In these cases, clients will often ship their hardware to me, so that I can integrate it locally. A very convenient way to work with clients in this way is to use a temporary import - it saves you the risk of paying taxes and duties for the expensive robotics parts, and as far as I know when doing a temporary import, you can keep the items for up to a year.

Travelling to the client’s location usually results in a couple of days of a hackathon solving issue. I love these, as it allows me to put the faces to Slack usernames, and I like the dynamics of these kinds of projects. On the flip side, the last time I did it, I ended up doing about 60 hours of work in 5 days - not very feasible, especially as you need to rest for a couple of days afterwards, but I would do it again!

Remote control of the client’s desktop is something that I try to avoid as much as possible when working with hardware. I find that not having physical access to the components is often very limiting (“can you please unplug the cable for me?”) and I estimate that in the worst cases, you are 20-40% less productive by working on robotics in this way.

Other Thoughts on Consultancy and Self-Employment

Freedom

Freedom is one of the most important aspects of what I do. If I don’t work with hardware for a project, then I can usually work from whatever place I want. If I have a day when I don’t feel like working, or can’t work, then taking a day off is not an issue. To an extent, I can choose whichever projects to work on (or at least to which I can say “no” to).

Having a possibility to go out into the mountains in the middle of the week is something I appreciate a lot

This freedom, however, comes with a price attached to it - if I’m not working, then I’m not making money. I don’t get bank holidays and if I get sick and can’t work, then I’m not earning either. These are the main reasons why I think working for oneself is not for everybody, especially when having no projects lined up, as it can become quite stressful.

Being self-employed means that you are at the centre of your business. For me, this means that my physical and mental health are my number one priorities. Examples of how I exercise these are:

Three times a week, around 4pm, I do sports (normally, I go to the gym, but during the pandemic, I’ve been going for a run)
Practising meditation as the first thing I do in the morning
In case of health issues, I look to fix them without thinking twice about the money (being located in Europe is a huge advantage here as I know no doctor’s visit will ruin me, financially)

Upwork

Over the past year, most of my work has come from Upwork (you can see my profile here). I won’t repeat the things I said last year - I’ve grown to appreciate the service. I realised that the 20% fee for the first $500 doesn’t hurt as much when you target long-term or high bid projects. The value I get in terms of not having to chase customers for payment, easily logging the time, and the discoverability, makes it worth it in my case. Some people criticise Upwork for the screenshot it takes of the screen while you are working for hourly projects, but for me, this is a non-issue - when I’m working on the project, I’ll never have anything private open and I’ll be just doing the work I charge customers for. If a project required me to have a webcam on though (it’s an optional requirement I believe), I wouldn’t accept it.

My hard-earned Upwork Top Rated Plus badge

If you are looking into starting on Upwork, then I’d recommend figuring out the best way to get paid, without losing out on payment and currency conversion fees. The most cost-efficient way I came across was setting up a TransferWise account (<– with this link, you’ll get your first transfer of up to 500 GBP for free). In TransferWise, I’ve set up a US account allowing me to handle ACH payments (the method Upwork uses to pay me). That way, I don’t pay anything to get the money from Upwork, but instead, I only pay for the TransferWise exchange rate, which is the best I found when I was looking into this.

To save time on browsing Upwork projects, I set up an RSS feed with the keywords that interest me. That way, I can see all of the projects that were posted on the platform, without having to go through the same ones by visiting the website.

Mastermind group

For over a year now, I’ve been having weekly Mastermind group meetings with my friend, Michał (if you need to tell a visual story, then I highly recommend getting in touch with him). In these weekly calls, we discuss the business and problems we are facing. Having someone to bounce ideas around is very helpful, especially if you can get honest feedback from someone running into similar issues as you do.

Me and Michał during one of our Mastermind meetings. Do you also think outdoor business meetings should be the norm?

Attention switching penalty

This is advice to all of you working on multiple projects at the same time. In my experience, every time I switch focus from one project to another, it takes about 10-20 minutes to get into the zone, and to not think about the other one. Because of this, I try to switch between projects as little as possible - ideally less than 2-3 times a day. Most of the time before a switch, I try to take a short break.

Closing thoughts

Having worked for over 2 years as a Remote Robotics Consultant, I can’t imagine doing anything else. I take lots of satisfaction and pride in my work, and I hope clients notice my focus on uncompromised quality. There is a small issue though - if your work is your hobby - when do you rest? I realised I might have been pushing myself a bit too hard during lockdown (having a clear split between the office and home was probably one of the reasons I was working from the office in the first place). Looking out into the future, I’m hoping to rent a proper workshop and work from there.

Do you have any questions about my work? Feel free to leave a comment and I’ll answer it! And if you would like to work with me, then feel free to send me an [email protected]">e-mail

Thoughts on 2 Years as a Remote Robotics Consultant was originally published by Mateusz Sadowski at msadowski blog on August 26, 2020.

Ouster OS1 - first impressions

2020-06-26T00:00:00-00:00

Recently I’ve started integrating Ouster OS1-16 on a client’s platform and he was nice enough to allow me to run some tests with this LiDAR sensor. This post describes my first impression with this unit and provides some pointers that can be useful for anyone looking into using Ouster sensors with Robot Operating System.

Hardware

Parameter	Value
Range	0.25-120m (at 80% reflectivity)
Range resolution	0.3 cm
Horizontal FoV	360°
Vertical FoV	33.2°
Rotation frequency	10 or 20Hz
Laser wavelength	865nm
Operating power	14-20W
Operating voltage	24V
Interface	Gigabit Ethernet (UDP)
Price	~3.5k USD

Setup

Here are some useful links that should get you started if you wanted to use OS1 sensors:

Ouster Resoures - Software User Guide is a must read before starting with this LiDAR
Ouster ROS driver - The driver developed by the Ouster team
ROS 2 Ouster Drivers - Drivers developed by Steve Macenski
Building Maps Using Google Cartographer and the OS1 Lidar Sensor - Very useful tutorial

Ouster OS1-16 measurements in Ouster Studio

Networking

All the tutorials mention setting up DHCP for Ouster sensors, however, if you are able to find the sensor’s IP address then in my experience so far you can use it directly. For these initial tests, I’ve used IP only and didn’t have any issues with the Ouster Driver, Ouster studio, and connecting to the sensor using Netcat.

I found netcat to be very useful in my first tests the commands that I was running were:

nc 10.42.0.245 7501
get_sensor_info
get_time_info
set_config_param timestamp_mode TIME_FROM_PTP_1588
reinitialize
write_config_txt

Among these commands get_time_info was very useful to roughly check the sensor timestamp against my computer time and make sure the clocks are synchronized.

Time syncronization

Handling time properly is very important, especially if you want to display your data live in software like RVIZ. Here are the timing options that you can find in ouster (via the Software User Guide):

TIME_FROM_INTERNAL_OSC - Use the internal clock. Measurements are time stamped with ns since power-on. Free running counter based on the OS1’s internal oscillator. Counts seconds and nanoseconds since OS1 turn on, reported at ns resolution (both a second and nanosecond register in every UDP packet), but min increment is on the order of 10ns. Accuracy is +/- 90ppm.
TIME_FROM_SYNC_PULSE_IN - A free running counter synced to the SYNC_PULSE_IN input counts seconds (#of pulses) and nanoseconds since OS1 turn on. If multipurpose_io_mode is set to INPUT_NMEA_UART then the seconds register jumps to time extracted from a NMEA$GPRMC message read on the multipurpose_io port. Reported at ns resolution (both a second and nanosecond register in every UDP packet), but min increment is on the order of 10 ns. Accuracy is +/- 1 s from a perfect SYNC_PULSE_IN source.
TIME_FROM_PTP_1588 - Synchronize with an external PTP master. A monotonically increasing counter that will begin counting seconds and nanoseconds since startup. As soon as a 1588 sync event happens, the time will be updated to seconds and nanoseconds since 1970. The counter must always count forward in time. If another 1588 sync event happens the counter will either jump forward to match the new time or slow itself down. It is reported at ns resolution (there is both a second and nanosecond register in every UDP packet), but the minimum increment varies. Accuracy is +/-<50us from the 1588 master.

Initially, I was trying to use TIME_FROM_SYNC_PULSE_IN with my ArduSimple boards. It worked OK but if you read the description of the mode above you’ll notice it says “If multipurpose_io_mode is set to INPUT_NMEA_UART then the seconds register jumps” - this explains why accuracy is +/- 1s from the pulse in source.

Another thing to note about using TIME_FROM_SYNC_PULSE_IN is that with 1.13 software there is a bug which causes the reported time to be off by 24 hours from the time provided.

If you need accurate timing then TIME_FROM_PTP_1588 is the way to go. Software User Guide has a very good workflow on how to set it up, even for a single machine. It’s best if machines support ethernet hardware timestamping but if it doesn’t the software timestamping seems to work well too.

Cartographer test

After making sure my launch files work with the sensor I’ve wanted to test some SLAM implementation. I ended up trying the sensor with Cartographer:

In my RVIZ settings, I’ve set the point cloud decay time to 4 seconds. As you can see the localization is not perfect but it’s not too bad, given I’ve used the default configuration from Ouster’s blog post on Cartographer integration.

Point cloud assembly

The second thing I wanted to discover for this blog post was the ways how to build a usable point cloud from the localized sensor data. This led me to point_cloud2_assembler from laser_assembler package. ROS Wiki describes this package as follows:

The laser_scan_assembler subscribes to sensor_msgs/LaserScan messages on the scan topic. These scans are processed by the Projector and Transformer, which project the scan into Cartesian space and then transform it into the fixed_frame. This results in a sensor_msgs/PointCloud that can be added to the rolling buffer. Clouds in the rolling buffer are then assembled on service calls.

Point Cloud Assembler data flow. Source: wiki.ros.org

To obtain this pointcloud I’ve added the following nodes to my launch file:

  <node type="point_cloud2_assembler" pkg="laser_assembler" name="my_assembler">
    <remap from="cloud" to="/os1_cloud_node/points"/>
    <param name="max_clouds" type="int" value="400" />
    <param name="fixed_frame" type="string" value="map" />
  </node>

  <node pkg="carto_demo" type="map_assembler.py" name="map_assembler"/>

where map_assembler.py is a simple Python script:

#!/usr/bin/env python

import rospy
from laser_assembler.srv import AssembleScans2
from sensor_msgs.msg import PointCloud2

rospy.init_node("map_assembler")
rospy.wait_for_service("assemble_scans2")
assemble_scans_srv = rospy.ServiceProxy('assemble_scans2', AssembleScans2)
pub = rospy.Publisher ("/assembled_pointcloud", PointCloud2, queue_size=1)

r = rospy.Rate(1)
last_time = rospy.Time(0,0)

while (not rospy.is_shutdown()):
    try:
        time_now = rospy.Time.now()
        resp = assemble_scans_srv(last_time, time_now)
        last_time = time_now
        pub.publish(resp.cloud)

    except rospy.ServiceException, e:
        print "Service call failed: %s"%e

    r.sleep()

Since I’ve ben saving all my data to a bag file I was then able to create an assembled pointcloud by running a bag_to_pcd node from pcl_ros with the following syntax:

rosrun pcl_ros bag_to_pcd <input_file.bag> <topic> <output_directory>

This command will create a .pcd file with the scan data that you can later import into software like CloudCompare:

Assembled point cloud in CloudCompare

Looking at the point cloud you can notice quite many outliers. Most of them are most likely the result of me not tuning the Cartographer for this post, however, the single line of measurements in the top left side of the above image looks quite a bit odd. Since it happens to be where a window in my apartment is I’m wondering if this could be caused by light interference.

Next steps

I’ll be running further tests on this sensor as I’m working on the client’s project. If I come across any significant findings I’ll make sure to update this blog post. If you would still like to know more then you can find two videos about Ouster on Steve Macenski’s Robots For Robots YouTube Channel.

Ouster OS1 - first impressions was originally published by Mateusz Sadowski at msadowski blog on June 26, 2020.

Enabling PPS on Jetson Nano

2020-04-28T00:00:00-00:00

I believe that in everyone’s life there comes a time where they need to do what they’ve always feared: they need to compile a Unix Kernel. This post is a step by step guide how I enabled the PPS support on Jetson Nano.

Background

For one of the projects I’m working on we wanted to have as precise timing information as it gets on a Linux board. In this particular project I opted on using gpsd together with PPS for driving the clock of Jetson Nano. During the research I’ve found a couple of dead ends and therefore I decided that I’d describe the process that I’ve followed in case anyone finds it helpful.

I’ve used the following process on my Jetson Nano Developer edition B01 with Jetpack 4.3 and a kernel version 32.3.1.

Useful links

Here are the links that I’ve found most helpful while researching the steps to enable PPS on Jetson Nano:

Jetson Kernel Customization - should contain the most up to date process
Ridgerun wiki - this guide might be slightly outdated but it helped me fill in the gaps
PPS on Jetson Nano - a topic from Nvidia Developer Forum
Enabling PPS on Jetson Nano with Jetpack 4.3 - a topic I started on Nvidia Developer Forum asking for support in enabling the PPS support

Tools setup

The paths I’ll be showing here are in the format they appear on my machine (Ubuntu 18.04), where my home directory is /home/mat. If you are following this tutorial make sure that you change the paths to match your setup.

Install the toolchain following these instructions
Download and install the SDK manager (I’ve used the version 1.0.1.5538 in this tutorial) After it’s downloaded, launch it and set up the development environment. In my case it looked like this:

SDK manager

Make sure that you select the Target Hardware to match your board. In my case it is Jetson Nano (Developer Kit version) (P3448)

In STEP 02 of the SDK I used the default settings and set the Target HW image folder as /home/mat/nvidia/nvidia_sdk. Then I pressed continue to download all the packages.

Configuring the kernel

These steps that I’ve performed should match the instructions you will find in the Jetson Kernel Customization page I posted before but I’d advise you to cross check.

Navigate to folder and create output kernel directory:

cd ~
cd nvidia
mkdir kernel_compiled

Export variables:

export CROSS_COMPILE=$HOME/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
export TEGRA_KERNEL_OUT=/home/mat/nvidia/kernel_compiled
export LOCALVERSION=-tegra

Sync the kernel repository by running source_sync.sh in /nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra. I synced to the tag tegra-l4t-r32.3.1
Build the kernel source configuration:

cd ~/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/sources/kernel/kernel-4.9
mkdir -p $TEGRA_KERNEL_OUT
make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig

Add PPS client support. In your editor of choice open the file .config in $cd $TEGRA_KERNEL_OUT directory (cd $TEGRA_KERNEL_OUT) and make sure the PPS is enabled as follows:

#
# PPS support
#
CONFIG_PPS=y
# CONFIG_PPS_DEBUG is not set

#
# PPS clients support
#
CONFIG_PPS_CLIENT_KTIMER=y
CONFIG_PPS_CLIENT_LDISC=y
CONFIG_PPS_CLIENT_GPIO=y

Add PPS gpio support. In ~/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/sources/hardware/nvidia/soc/t210/kernel-dts/tegra210-soc edit tegra210-soc-base.dtsi and add the following lines:

pps {
    gpios = <&gpio TEGRA_GPIO(B, 7) 0>;

    compatible = "pps-gpio";
    status = "okay";
};

Building and flashing the kernel

Build the kernel:

cd ~/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/sources/kernel/kernel-4.9
make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j8

Prepare the board for flashing. In case of board B01: make sure the board is powered off, short pins SYS_RST and GND to enter recovery mode, power up the board from the power supply and connect the board to the USB port of your machine
Flash kernel

cd ~/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/kernel
cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image Image
cd dtb
cp -a /$TEGRA_KERNEL_OUT/arch/arm64/boot/dts/. .
cd ~/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra
sudo ./flash.sh jetson-nano-qspi-sd mmcblk0p1

After the previous step you should see the board flashing. After the flashing is done you will be able to set up the system using a computer screen, keyboard etc.

Testing pps

Make sure ls /dev/pps* returns /dev/pps0 and /dev/pps1
Install pps-tools sudo apt-get install pps-tools
Run pps test on /dev/pps0: sudo ppstest /dev/pps0 You should see the output like this, with no PPS device connected:

trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1588064315.861849703, sequence: 4355 - clear  0.000000000, sequence: 0
source 0 - assert 1588064316.885912540, sequence: 4356 - clear  0.000000000, sequence: 0
source 0 - assert 1588064317.909913941, sequence: 4357 - clear  0.000000000, sequence: 0
source 0 - assert 1588064318.933846094, sequence: 4358 - clear  0.000000000, sequence: 0
source 0 - assert 1588064319.957913486, sequence: 4359 - clear  0.000000000, sequence: 0
...

Now, to test the /dev/pps1 device you will need to connect your PPS source to pin 18 of the 40-pin header of the Jetson Nano (probably connecting ground pins between the PPS source and the Jetson won’t hurt either). By running a ppstest again on /dev/pps1 you should see a similar output as we had with /dev/pps0

ArduSimple simpleRTK2B and Jetson Nano, almost ready for PPS triggering

That should be it! The setup you can see in the above picture worked well for the PPS input after connecting the antennas and powering everything up. Now, with some NMEA sentences being sent from the simpleRTK2B to the Jetson and some gpsd and chrony configurations I should be able to drive the clock of the Jetson.

Hope these steps will save you a bit of time looking for the PPS solution, especially if like me it was the first time you were building a Kernel.

Source: XKCD

Enabling PPS on Jetson Nano was originally published by Mateusz Sadowski at msadowski blog on April 28, 2020.

360 camera for industrial inspection with ROS and Robosynthesis robots

2020-04-23T00:00:00-00:00

Together with Robosynthesis we’ve been working on developing and integrating a 360 camera module for our ROS based industrial inspection robots. I was so pleased with the results that I couldn’t pass on the opportunity to share what we’ve been up to.

Background

In one of the projects we have worked on we run into an issue - teleoperating a robot in an environment filled with obstacles was really difficult with the cameras we had at the time. In that particular project we ended up with the following camera setup:

One camera was pointed forward
Two cameras were pointing to the sides
Fourth camera was attached at the top of the robot and looking down, allowing the operator to get a good view of what’s next to the robot’s wheels

The user interface for this setup looked something like this:

Not the actual interface

Each of the above boxes was providing the user with the camera view. This worked well enough but wasn’t very intuitive. Months after that we’ve started exploring what we’ve called a helicopter 360 view.

4 rectified camera images almost ready to be stitched together

We have managed to come with the solution that allowed rectifying images from four cameras simultaneously. The idea at the time was that if we get a high enough field of view we should be able to create a nice top down projection of the environment around the robot. There is definitely a value in a top-down projection around the robot but why constrain yourself to 2D view when you can go full 360?

360 camera ROS module in action

Before we jump into the technical details of the solution let me show you how it works:

In this video I’m able to use my mouse to look around the 360 sphere that has the camera feeds overlayed. The interface is very intuitive - you just click and drag the mouse to look at the environment and use the scroll wheel to zoom in or out. There are couple of things I like about this solution:

Coverage - compared to the alternative solutions we are looking everywhere around the robot at the same time. This is great for inspection because we can record the data and play it back later on allowing us to focus on many points of interest
Ease of use - this solution is way more intuitive to the operator than our previous 4 separate camera views
Manageable latency - you can actually operate a robot using this view as the only camera feed

The first thing to get to our solution was to get USB cameras with a high field of view. We ended up with 220 degrees FoV USB cameras. The first test was getting the image feed from a single camera:

Hello World! (That's a proper fisheye view, isn't it?)

Then we entered a very rapid prototyping stage with this handy prototype:

It took about 5 minutes to assemble this one

This first prototype allowed me to quickly prove the concept and prepare and test the software stack. In less than a week from the first proof of concept the team took it to the next level:

Final result mounted on the robot

In this short period of time we have:

Designed the casing
Added a flood light that can be triggered with a ROS service call
Made it into a module that can be plugged in anywhere on the robot’s deck

Let there be light

Where there is robot there is light

Technical details

I hope that at this point you are wondering how we make this 360 view. The core piece of software that we’ve used is the RVIZ Textured Sphere open source plugin developed by Researchers from the Nuclear and Applied Robotics Group at the University of Texas. The plugin takes the image feed from two sources and then applies them onto a 3D Sphere.

The original rviz_textured_sphere demo. Credit: Nuclear and Applied Robotics Group

Since in the first prototype we have used USB cameras we have used standard UVC drivers for ROS. Unfortunately the cameras we used were suboptimal, providing a rectangular image for a circular image view. Because of that we had to post process the images to be square and the fisheye view to appear in the center.

Since we want the 360 camera view to be used by the operator, the hard requirement for us is a latency below 1 second. In this first prototype we saw around 500ms of latency with the 800x600px resolution for a single camera. In the next iteration I’d like to drive the latency down below 300ms while increasing the camera resolution. Stay tuned for the future updates!

As a bonus please find the below picture showing two Robosynthesis robots with heaps of modules being ready for industrial inspection.

Modular robots, half of them carrying our 360 camera module

360 camera for industrial inspection with ROS and Robosynthesis robots was originally published by Mateusz Sadowski at msadowski blog on April 23, 2020.

ArduSimple RTK - ROS integration

2020-04-05T00:00:00-00:00

As you might have seen from two of my previous posts (1, 2) I have been doing some testing with ublox F9P based ArduSimple RTK setup. In this blog post I’m describing how I integrated these modules with Robot Operating System (ROS).

Introduction

Please see my first post in this series to learn more about my ArduSimple setup the short story is I use simpleRTK2B for my base station and simpleRTK2B+heading for the rover.

ArduSimple RTK base (left) and rover (right)

Configuration

Since I managed to break the whole configuration by overwriting it with the ublox ROS node by using a wrong config, I had to fire up the u-center application. From ArduSimple Github Repo I’ve downloaded the following configuration files:

srtk2b_base_FW_HPG112.txt - base module
srtk2b+heading_lite_movingbase_FW_HPG112.txt - the heading lite module (the small board), flashed by connecting to the XBEE USB
srtk2b+heading_rover_F9P_FW_HPG112.txt - the main rover module

To flash the config files I’ve followed instructions on ArduSimple website. When writing the configuration file pay attention to the firmware version and if needed follow these steps to update your modules.

When flashing the ublox configuration following the above guides the checkbox for storing the configuration files was disabled. My assumption at the time was that F9P will automatically store the settings in the memory, however it turned out I was very wrong on that and I’ve noticed this only after switching back and forth between Windows and Linux numerous times.

To save the loaded config you will need to save it in View->Configuration View->CFG, Select all Devices and press the Send button at the bottom of the window.

Saving u-blox F9P configuration

U-blox ROS driver

When searching for the ROS driver the most important feature I was looking for was the support of UBX-NAV-RELPOSNED messages that would provide the relative heading between the two rover antennas.

My ROS package of choice that would support F9P became ublox by Vijay Kumar Lab. As I noted in the previous section using this package I’ve overwritten the modules configuration by running some of the example launch files in the repository. It took me a while to realise the misconfigurations in the modules but when I did I made sure to have the option config_on_startup: false in the .yaml file for my rover config. You will find the yaml file that I used in my experiments below:

debug: 1 # Range 0-4 (0 means no debug statements will print)

device: /dev/ttyACM0
frame_id: gps
config_on_startup: false
uart1:
  baudrate: 115200
rate: 1
nav_rate: 1
publish:
  all: true

The ArduSimple launch file I used looked as follows:

<launch>
  <arg name="node_name" default="gps"/>
  <arg name="param_path" default="$(find ardusimple_rover)/config/ardusimple.yaml" />
  <arg name="output" default="screen" />
  <arg name="respawn" default="true" />
  <arg name="respawn_delay" default="30" />
  <arg name="clear_params" default="true" />

  <node pkg="ublox_gps" type="ublox_gps" name="$(arg node_name)"
        output="$(arg output)"
        clear_params="$(arg clear_params)"
        respawn="$(arg respawn)"
        respawn_delay="$(arg respawn_delay)">
    <rosparam command="load" file="$(arg param_path)" />
  </node>
</launch>

After launching the above file I could echo /gps/fix topic for the GPS messages. When you do that in the RTK setup pay attention to the covariance fields, if they are very low then you most likely have a great view of the sky and the base unit is providing good corrections. To see the relative heading between the two modules you can check the /gps/navheading topic.

Robot Localization

In this blog post I wanted to provide the simplest example that you could run just with the RTK hardware and no other sensors. As a test I’ve created a basic setup for robot_localization and navsat_transform_node to create a working TF structure. The only data I was fusing in the EKF was the GPS fix message and the relative heading between the modules.

Because I only have information from GPS this setup does not adhere to REP-105 - we don’t have any source of odometry that would be continuous. That’s why you usually use GPS as one of the inputs providing a map->odom transform (more on this in the last section).

There are two nodes that I used here:

navsat_transform_node - to produce an odometry message from gps fix
ekf_localization_node - to create a transform between the odom and gps frames

You will find all the configuration files that I’ve created in ardusimple_rover repository. There are two things to note here: because the observed rotation between the modules (/gps/navheading topic) is relative the navsat_transform_node will have an error in orientation. Normally you would use an IMU instead which should produce the correct results. The only two topics that are fused in the ekf are /gps/fix and /gps/navheading - in a real setup you will want to use information from your other sensors.

Results

Due to the lockdown I was only able to perform a test in a residential area, driving a car with two rover antennas attached to the roof. I’ve placed the base antenna in a fixed position and let it run for a couple minutes. After manually adjusting the yaw_offset parameter of navsat_transform_node to align odometry with true gps coordinates I was able to get the following result in mapviz:

Mapviz + U-blox ArduSimple in action (4x)

Above you can see the following information:

blue dots - raw GPS information
green line - filtered odometry
yellow points - location of gps frame in odom frame

In the graph below you can see horizontal gps position covariance (top) and the orientation covariance from F9P. As the car moved from an area with a wide sky view to a more urban area you can see an increase in covariance values.

Covariance values for the whole test run, courtesy of PlotJuggler

Next steps

The setup I described together with the demo repository should get you started on integrating and testing ArduSimple RTK boards (probably anything based on U-blox F9P too). If you wanted to integrate this solution on a mobile robot then you would run two instances of robot_localization:

ekf map->odom - fusing information from all of your sources, including RTK + heading from RTK
ekf odom->base_link - fusing only continuous sources of odometry (wheel odometry, IMU)

For more information on how to pull this off you can see the robot_localization wiki.

ArduSimple RTK - ROS integration was originally published by Mateusz Sadowski at msadowski blog on April 05, 2020.

ArduSimple RTK - preparing for testing

2020-02-06T00:00:00-00:00

In my previous post I was sharing my first impressions from using ArduSimple RTK GPS. It’s time to share some information about the small but steady progress I had made since!

Introduction

In the last post I was testing the RTK in a very static manner by putting the GPS antennas next to the wall of a building. I’m not a fan of sticking electronic boards straight onto any rough surfaces that are unprotected in any way. My first thought on securing the electronics bits while was to 3D print this case. The issue with this case is that it’s not high enough to fit the rover module that has an extra GPS board that sits below the XBee radio.

ArduSimple RTK base (left) and rover (right)

My next thought was that I could perhaps design my own case but instead I decided to design a base plate that can hold the module. If it’s of any use for you and would like to 3D print it then you can grab it here. You will just need to secure the modules to the plate using 3 M3x8 screws. I recommend choosing screws with as small head as possible - the screw that goes right below XBee module is very close to the pin headers.

3D printed base plate for ArduSimple RTK modules

Unfortunately the base plate won’t help with any environmental protection therefore behold…

Project Boxes

Shiny new project boxes

I figured that purchasing two project boxes will be way faster than designing the case from scratch and I figured I will drill two holes on every side of the box to run the necessary wires through and will be done in no time. Also the these project boxes are (were) IP66 rated.

Almost done here (don't think they are IP66 anymore though)

After cutting the holes in the boxes it was time to place the RTK modules inside:

Do you see any issues here? Me too! Most of the cables are not as flexible as I had hoped and there is simply no way to acces the USB ports on the left side of the modules (the ones I was actually planning to use).

Here are the boxes with antennas attached. You can clearly see that there is no way to get to USB ports in this configuration. Well… sometimes you’ve got to do what you’ve got to do…

From now on you can call me project boxe butcher

At this point I’m pretty sure design and printing my own case would be a faster option (mostly due to the lack of proper tools). The good news is this should do the job. If it starts raining in the field the covered box should give me enough time to grab to extract it from the field without destroying the hardware.

Lessons learned

Doing a bit of manual hardware work was a good break from all the software I’ve been doing in the past couple of months. Here are some things that I’ve learned and hopefully they are useful for you too:

Designign a base plate before I had the boxes - in hindsight I should’ve waited for the boxes to arrive before desiging a base plate. That way I would have more control over the position of the modules within the box and could cut way more precise (and smaller) openings in the boxes
I should’ve ordered a box with a transparent lid that way I could see the RTK status lights without needing to open up the box
When manipulating the XBee antenna sometimes the XBee lifts up, ideally antenna should have any ‘wiggle room’

Next steps

I’m happy enough with the state of my project boxes to start running some experiments. Since the ArduSimple RTK modules are based on Ublox ZED-F9P I’m going to start with ROS ublox package. I just need to set up project box for my Raspberry Pi or… not this time!

ArduSimple RTK - preparing for testing was originally published by Mateusz Sadowski at msadowski blog on February 06, 2020.

ArduSimple RTK2B+heading - first impressions

2020-01-06T00:00:00-00:00

ArduSimple provided me with their RTK2B+heading module recently. In this blog post I’m describing my experience running the first tests with these modules and my planned further tests. My end goal is to integrate the modules with ROS and get a precise positioning in 2D with extra heading information.

RTK basics

They say that a picture is worth a thousand words, so here is a 5 minute YouTube video that should give you basic information on RTK in case you haven’t heard of it before:

What is special about the module that I’m testing from ArduSimple is the rover board that comes with 2 GPS chips, allowing to plug in two GPS antennas and figuring out the rover heading based on available information.

Setup

The setup I’ve received from ArduSimple consists of the following:

simpleRTK2B+heading - rover module
simpleRTK2B - base station module
Long range radio module x2
3x u-blox ANN-MB-00 GNSS multiband antenna

ArduSimple base module (left) and rover module (right)

The ArduSimple boards are based on u-blox ZED-F9P, a professional grade multiband GNSS modules.

The long range radio modules that I’ve received are XBee SX 868 - an 863-870 MHz radio modules boasting a line of sight range of 14km.

u-blox ANN-MB-00 antenna

The u-blox antennas are IP67 rated have two mounting holes and an embedded magnet that makes it convenient to attach to metal surfaces.

Testing

One advice from my drone days that stuck with me is to always plug in all antennas before powering the system. That’s what I did when working with this system and I recommend that you do that too.

running u-center software

At this point I only run some static tests. The first test I performed using u-center software, even when I put the antennas outside of my window in an urban space (conditions far from ideal due to multipath) the software was reporting 0.23m accuracy in 2D - not bad at all!

Streaming GPS information to Google Earth

Thanks to the advice from the ArduSimple team I was able to display the position on Google Earth (to do that select Menu Bar > File > Database Export > Google Earth server in u-center).

As another experiment I tested the receivers with ROS-Agriculture ublox_f9p node. Amazingly it worked out of the box streaming out the position data.

After I’ve performed the tests I’ve found out about ublox package that seems to implement RELPOSNED message and hence should allow me to stream heading from my rover setup.

Summary and next steps

I found the ArduSimple units to be a great piece of kit. I still have quite a bit of work to do to implement them on a robotic platform, here is a rough plan how I’m going to pull this off:

Build cases and a base station setup (ideally so that it can be placed on a tripod)
Test out the ublox package
If the package from previous point is not satisfactory: fork ROS-Agriculture ublox_f9p repository and make sure UBX-RELPOSNED message is handled and the driver publishes heading
Capture data by walking around with the module in hand
Integrate the module on my mobile robot development platform and tune the EKF to work with the RTK module
Enable corrections via NTRIP

ArduSimple RTK2B+heading - first impressions was originally published by Mateusz Sadowski at msadowski blog on January 06, 2020.

Giving LaMa a shot

2019-11-29T00:00:00-00:00

If I knew how to draw I would create a header for this blog post that would show a lama getting vaccinated. Since you won’t be seeing it any time soon, let’s jump into iris_LaMa, a localization and mapping library that works with ROS.

Setup

Robosynthesis development platform

Similarly to my previous experiments (T265 test and slam_toolbox evaluation) I’ve run the test described in this post on a Robosynthesis development platform.

The only difference from the previous posts is that I added an IMU, therefore the fused robot odometry should be a tad bit better. Highlights of the setup are as follows:

Robosynthesis differential-drive mobile robot development platform
RPLidar A1 (hobby grade, scanning at ~7Hz)
Onboard computer running ROS Melodic and iris_lama (commit: 07808d87) and iris_lama_ros (commit: 54df5359)
Pixhawk autopilot used as an IMU

First LaMa impressions

LaMa is the library developed by Intelligent Robotics and Systems (IRIS) Laboratory at the University of Aveiro. The package is maintained by Eurico F. Pedrosa and is open sourced under BSD 3-Clause licence.

Getting started with LaMa was very simple - pull the two repositories, build them, change couple of parameters (scan_topic, global_frame_id, odom_frame_id, base_frame_id) to match my platform and run it.

There are 3 nodes of interested in the ROS package:

slam2d_ros - online SLAM
loc2d_ros - localization in the known map
pf_slam2d_ros - particle filter slam

In this blog post I’ll focus on the first two nodes.

Online SLAM

SLAM with LaMa

The first impression I got using this SLAM package was that it “just works”. Out of the box without tuning any parameter values except for the frame name changes the SLAM seemed to perform very well. For the full parameter list you can see the project readme. I found that the number of parameters you can tune is smaller than in case of other packages I used so far (looking at you Cartographer). The good thing about it is that you don’t need to spend so much time tuning the parameters to get good results, on the other hand you might not have so much flexibility.

Localization

With the localization node you can start a map_server and the localization node will work with the map coming from the server. Currently the node doesn’t support global localization (as per Readme.md) but instead you can send the robot pose on the /initialpose topic (this means that if you set the pose estimate from RVIZ it will work too). I found that even if you set the pose ~1-2m from the real robot pose it converged to the true pose as the robot is moving around.

Apples to oranges

Since I already covered slam_toolbox in one of my previous posts I’ve decided I would try to see how they compare. Do you think arbitrary comparisons of two SLAM results is meaningless? Me too! But I’m going to do that anyway.

Above you will find a video comparison of online SLAM with iris_lama compared to slam_toolbox. I run the two solutions with default configs and only changed the platform related settings (mostly the names of frames and LiDAR range). Here are some of my observations on LaMa:

Visually the position estimate of LaMa looks a bit smoother (less discrete jumps)
Erroneous readings from my LiDAR are added to the map in LaMa (you can see points appearing way outside of the map)
The very small features seem to be missing from the map (you can see that well in the upper right corner of the map)

It’s quite possible that some of the things above can be tuned through parameters, however I didn’t find any obvious parameters described in the readme file.

Another thing I wanted to know is how does the position estimates compare. For this reason I have used the evo Python Package and described my workflow in full in my previous post. In the screenshots below I have used the output from iris_lama as a ground truth:

Relative Pose Error results on my data

What these images tell us is that most of the time the output of both packages is quite close to one another (you can right click on the image to see it in full size). The issues with this comparison is that without ground truth we can’t say anything more about it, which brings me to the next section.

CPU usage

I think the biggest advantage of iris_lama is its low CPU usage (In the project readme author says that it runs great on Raspberry Pi 3B+). Here is the comparison of CPU usage of iris_lama (left) and slam_toolbox (right).

CPU usage comparison between the two packages

You will see that the LaMa CPU usage peaks at around 15% and slam_toolbox at 450%. I hope that one day I will get to repeat this experiment in a larger environment and see how both packages manage it. I grabbed the CPU usage of both packages using cpu_monitor package and running it along the bag file with raw data and each SLAM package. Then I plotted the output using PlotJuggler.

Links

Here are some links that can provide you with further reading or allow you to replicate my results.

Next steps

I’ve recently received a simpleRTK2B+heading module from ArduSimple which should get me sub centimeter level position accuracy. This should be a decent source of ground truth for my follow up experiments. Stay tuned for more information in the next post!

Giving LaMa a shot was originally published by Mateusz Sadowski at msadowski blog on November 29, 2019.

Comparing SLAM results using ROS and evo

2019-11-21T00:00:00-00:00

While working on another blog post I fell into a rabbit hole of comparing SLAM packages using ROS. This blog post briefly describes how I managed to compare the results of two SLAM packages by using evo Python package.

Background

Recently I’ve started working a lot with various SLAM packages in ROS using the Robosynthesis dev platform. I figured that opening RVIZ, displaying the robot model in the map frame and saying “yeaah, that looks about right” might not be necessairly the best way to evaluate the quality of SLAM.

What my data lacks is groundtruth as at the moment I don’t have a way to capture it. This post describes the way to evaluate output of two SLAM methods against one another.

evo

I found evo, while doing research for my newsletter Weekly Robotics and featured it in issue #58. As per the description on project website:

This package provides executables and a small library for handling, evaluating and comparing the trajectory output of odometry and SLAM algorithms.

The thing most interesting for us in the scope of this blog post is that it supports bag files!

Gathering the data

To gather the data I run the robot around my office with all my mapping stack running and made sure that the robot surveyed enough area to build a good map. The next step was to run 2 SLAM packages using the data from my .bagfile. To do it correctly you need to ensure your bag contains only the data of interest, hence rosbag filter comes handy:

rosbag filter slam.bag bag_filtered.bag "topic == '/clock' or topic == '/rr_robot/joint_states' or  topic == '/odometry/filtered' or topic == '/scan' or (topic=='/tf' and m.transforms[0].header.frame_id != 'map' or topic=='/tf_static')"

The most important things we need from the bag are clock, laser scan topic and the whole tf structure except the map->odom transform if your setup follows REP-105.

After these steps we shold have a set of raw data that we can use to run some SLAM packages from the recorded input. The main reason I find it useful is that it allows tuning the parameters to some extend and rapidly visualise the change in quality.

Rebagging the data

The next step that I performed was to run the slam package of choice with the recorded data. Couple of pointers:

Make sure to set use_sim_time parameter to true
Run your bagfile with –clock argument
Visualise everything in rviz to make sure your data is working correctly. I advise displaying the map and pose of the robot

When running the SLAM node make sure that you record the bag file (let’s call it run_1.bag) of your run, this time record all the data as we won’t be filtering this again.

After you have grabbed the run_1 you can re-do the experiment with another package and create run_2.bag file.

Extracting the data

Evo currently works only with geometry_msgs/PoseStamped, geometry_msgs/TransformStamped, geometry_msgs/PoseWithCovarianceStamped and nav_msgs/Odometry topics. That means that in case of most SLAM packages that I used to date you can’t use it directly as most of them provide map->odom tf transform.

Luckily evo repository contains a file we can use for our needs. In short this script will look at the bag file and will create a pose topic between two frames of interest. This is exactly what we need now and we will run it twice on our run_*.bag files:

WARNING! The methods below are destructive, make sure you keep copies of your original bag files.

./record_tf_as_posestamped_bag.py  --lookup_frequency 15 --output_topic run_1_pose --bagfile run_1.bag map base_link

./record_tf_as_posestamped_bag.py  --lookup_frequency 15 --output_topic run_2_pose --bagfile run_2.bag map base_link

The above scripts assume that your map frame is called ‘map’ and the base_link is a reference point on your robot and is called ‘base_link’. Running the script our bag file will only contain a single topic named run_*_pose.

Merging bags

At the moment it is not possible to run evo with two bagfiles as input therefore we will need to merge them. The way I did it was by running this script:

./bagmerge.py run_1.bag run_2.bag -o merged.bag

Working with evo

Having the data with two topics (run_1_pose and run_2_pose) we can now get some metrics on our solutions using commands like:

‘evo_ape’ - absolute pose error
‘evo_rpe’ - relateive pose error

You would run them as follows: evo_rpe bag merged.bag run_1_pose run_2_pose --plot and you would see the following output:

Relative Pose Error results on my data

What I’m showing here is probably a tip of the ice berg, for more information about evo please see the wiki.

Results discussion

The most important thing I’m lacking in my comparison is the groundtruth. Without it we can’t be certain which of the results is more correct but if you had an access to a system that could provide a groundtruth this approach should be working very well and you should be able to use groundtruth straight out of the box.

I think that the process I described in this blog post is a bit complex at the moment but it should be quite easy to streamline it. If you have some ideas how to improve it let me know!

Comparing SLAM results using ROS and evo was originally published by Mateusz Sadowski at msadowski blog on November 21, 2019.

Intel Realsense T265 tracking camera for mobile robotics - first impressions

2019-10-15T00:00:00-00:00

In this post I will write about my first impressions after working with Realsense T265 on a wheeled mobile robot and give some tips on the configuration I think is most correct with respect to ROS standards.

Setting up

Robosynthesis development platform

I’ve tested the T265 on a Robosynthesis Dev Platform that you might have seen it in my previous post. While working with the T265 tracking camera I spent a fair bit of time going through the documentation and ROS package source code, hope that any of the insights I describe in this post will help you get started.

There are two sets of software we will be interested in in this blog post:

realsense-ros package (GitHub) (while working on this post I was on da4bb5d commit hash)
librealsense (GitHub)

There are some inconsistencies between the above packages and I’ll do my best to clear them up in this post. Please note that the T265 seems to be in an active state of development therefore some of the information contained in this post might change over time (I’ll do my best to keep it up to date though).

Coordinate frames

Coordinate frame setup is something I had the most issues with when I first started with T265 with realsense-ros package. First let’s look at the T265 frame (as seen in librealsense).

T265 coordinate frames

T265 uses a VR coordinate system which differs from the one you would expect in ROS. Fortunately the realsense-ros handles the tf transforms for us, making our lives a bit easier. Here is what we need to keep in mind:

Any frames with a word optical in them (e.g. camera_fisheye_optical_frame) are the same frames are the T265 frames in VR coordinates
The camera and IMU messages are delivered in the optical frames
The parameter pose_frame_id in the realsense-ros differs from the librealsense pose frame. Its orientation follows ROS convention (x-front, y-left, z-up in the global frame)
pose_frame_id in realsense-ros is the location of the camera in the odometry frame. There is a static transform between the pose_frame_id and the base_frame_id of the camera (not to be mistaken with the base_frame in the ROS traditional sense)

Below you can see an example ROS tf tree that the realsense-ros can provide for us:

slam_toolbox RViz plugin window

You should be able to get the same output by setting the following parameters in the launch file (pseudocode):

camera = “rs_t265”
tf_prefix = “$(arg camera)”
publish_odom_tf = “true”
odom_frame_id = “odom”
base_frame_id = “$(arg tf_prefix)_link”
pose_frame_id = “$(arg tf_prefix)_pose_frame”

Shortcomings

There are two things to note about the above tf structure (most of them coming from REP-105)

A tf frame can have only one parent. This means that if you have a base_link frame specified in your robot description you won’t be able to directly specify a transform for base_link->rs_t265_link or base_link->rs_t265_pose_frame as this would break your tree
By convention the measurements in the odometry frame have to be continuous (without discrete jumps) this means that if you were to use the setup described above then you would need to set “enable_pose_jumping” parameter to false (GitHub issue). More on this later

The most ROS-proper setup I can think of

Here are some of the considerations for creating a most proper setup with ROS package for Intel Realsense T265 that I can think of.

First of all, I like my sensor frames being relative to the base_link frame of my robot platform. Therefore in my urdf description I would define a static base_link to the camera pose_frame (the ROS one). To have this working we need to set “publish_odom_tf” parameter to false (this way we ensure that the camera pose_frame has a single parent, a base_link).

Say we would like to use the camera as a source of odometry. My suggestion for getting there is to:

Run a robot_localization node that will listen to the odometry message from T265 and publish an odom->base_link transform. That way we can easily ensure that the tree is continuous
Set enable_pose_jumping parameter to false so that the pose of the robot in odometry frame is continuous (WARNING read this section until the end before implementing it since it might cause significant errors). Some of the packages you might be using might make some assumptions following REP-105 so better safe than sorry
Optional: create a t265 filter node that will change the odometry frame_id from the pose_frame to base_link. That way you should be able to directly compare the various odometry sources on your platform (at least comparing the wheel odometry against the t265 odometry sounds like an interesting experiment)

I successfully run some tests with the first two points from the above list and I was quite satisfied with how ‘clean’ the setup was w.r.t. ROS good practices. Because of velocity drift that I observed on multiple occasions in my setup I stopped looking into it.

Wheel odometry

According to the docs T265 absolutely requires wheel odometry for robust and accurate tracking. To provide the odometry information you will need to:

Specify the topic_odom_in parameter
Create a file with odometry calibration

The first requirement is trivial; you just need to make sure that you correctly specify the topic name.

The fields you need to fill in in the calibration file are T and W vectors in extrinsics field (you will find an example in this comment). What you need to keep in mind is that T is the translation from camera pose frame (in the sensor/VR frame, not the realsense-ros pose frame) to your base_link and W is the rotation between these two frames in axis-angle representation. You will find some useful information about this in this pull request.

Closing thoughts

I really like the idea behind the Realsense T265. Having an affordable sensor that can be easily integrated onto any robot would be a great thing to have. I think the T265 is going in the right direction, however I would let it mature before using it on a commercial system but I think it will get there and will provide a true ‘plug&play’ ROS experience, adhering to good ROS practices.

Is there anything that I missed? Your feedback is highly valued so feel free to leave a comment! I’ll be following the T265 development and try to update this post as needed.

Update

It has been almost 1 year since I wrote this blog post. I was recently looking to integrate it on a robot. Unfortuantely there are two problems that prevented me from using it on the robot:

As you will see in the comments Intel is not planning to address any existing or new issues on T265 as they shifted focus to the new products. If you are making robots that have to run over long period of times with high accuracy I would advise to wait for the new products to come out before investing in T265.

Intel Realsense T265 tracking camera for mobile robotics - first impressions was originally published by Mateusz Sadowski at msadowski blog on October 15, 2019.

Hands on with slam_toolbox

2019-09-18T00:00:00-00:00

In the past couple of weeks, as part of a project with Robosynthesis, I’ve been exploring slam_toolbox by Steven Macenski. This post summarizes my experience so far.

Here is the description of the package taken from the project repository:

Slam Toolbox is a set of tools and capabilities for 2D SLAM built by Steve Macenski while at Simbe Robotics and in his free time. This project contains the ability to do most everything any other available SLAM library, both free and paid, and more. This includes:

Ordinary point-and-shoot 2D SLAM mobile robotics folks expect (start, map, save pgm file) with some nice built in utilities like saving maps

Continuing to refine, remap, or continue mapping a saved (serialized) pose-graph at any time

life-long mapping: load a saved pose-graph continue mapping in a space while also removing extraneous information from newly added scans

an optimization-based localization mode built on the pose-graph. Optionally run localization mode without a prior map for “lidar odometry” mode with local loop closures

synchronous and asynchronous modes of mapping

kinematic map merging (with an elastic graph manipulation merging technique in the works)

plugin-based optimization solvers with a new optimized Google Ceres based plugin

RVIZ plugin for interating with the tools

graph manipulation tools in RVIZ to manipulate nodes and connections during mapping

Map serialization and lossless data storage

… more but those are the highlights

Our test setup

Here are the highlights of our setup:

Robosynthesis differential-drive mobile robot development platform
RPLidar A1 (hobby grade, scanning at ~7Hz)
Onboard computer running ROS Melodic and slam_toolbox (commit: 9b4fa1cc83c2f)

Robosynthesis development platform used for slam_toolbox evaluation

First impression

The first test we’ve run was with the default parameters that come with slam_toolbox with minor changes (frame names, and max laser range).

In ROS, as a good practice, we usually have a TF tree setup in the following way (at least as a minimum when doing SLAM):

map -> odom -> base_link

If you would like to know more about the transforms then REP-105 is your friend. Sometimes we run a competition in the office who can recite it faster!

Our expectation of slam_toolbox is to provide us with a map -> odom transform. In our tests we’ll use odom -> base_link transform from wheel odometry. Is it going to drift? Yes, it will! But slam_toolbox will have our back!

Here is a short gif showing our first test, driving the robot at a reasonable speed (at least for an indoor robot) around an office:

Performing SLAM with slam_toolbox (replay with 3x rate)

And if you’d like to see some of the raw data used during the above session then you can download the bag file here.

Taking it further

What you’ve seen in this blog post are only the first trials with this SLAM package. If you take a look at the configuration files you will see that there are lots of parameters that can be tuned.

A very helpful tool that comes with slam_toolbox is the RViz plugin (Panels->Add New Panel->slam_toolbox->SlamToolboxPlugin)

slam_toolbox RViz plugin window

This blog post is only the beginning of our adventure with slam_toolbox but we’ve liked it so much that we decided to share the results with you. Stay tuned for more information about the hardware and open source that we use!

Hands on with slam_toolbox was originally published by Mateusz Sadowski at msadowski blog on September 18, 2019.