Let me be honest: it is a joy to look at all of this again. Just like that, retro. To see what we did, how we did it, and why. Well, ok, the why. That one is sometimes hard.
Just like that, retro
After the object recognition model of 2020 we had something that worked: a phone recognises things and hangs labels in the room. And we had a question that grew out of it. If the phone knows where a plant stands, why does it not know what the room looks like?
The question became a new attempt. It was called Scan Your World, again from Zug, from devpoint, and it was in the App Store in 2021. The website is still online, scanyourworld.devpoint.org, with prices, a roadmap and the sentence "There will be more coming in 2021". I opened it again yesterday. You read that and are proud and a little embarrassed at the same time.
From recognition to the room
Object recognition says: there is an oven. Scan Your World says: this is what the room looks like in which the oven stands. In three dimensions, with surfaces, with measurements, and with everything you want to attach.
The step sounds small. It was not. Recognising is a model looking at an image. Scanning is a phone moving through a room, receiving thousands of points per second from the LiDAR, needing to know where it is right now, and building from all of that a model that still holds up afterwards. That is not a model. That is mathematics, geometry, and a lot of operations.
What the app could do
Three modes. In 3D model mode you scan, the app builds the model on the device, and afterwards you work inside it: measure, calculate areas, set notes, draw on surfaces, save, share. In live mode the screen is a window into the room, you place notes and drawings right where you stand, a bit like with AR glasses, only with the phone in your hand. And a third mode we called "pet mode" was on the roadmap: see points live, see quality, go back to weak spots and rescan.
Three scan types. Full screen, everything the camera sees. Sphere, only what lies inside a sphere you place and size yourself. Bubble, everything around you, you are the bubble, you walk, the bubble walks with you. The website says: "The bubble is you. You are the bubble. Walk the bubble." We really wrote that.
And measuring around the corner. You draw a line across a surface, the app follows the surface, even over an edge, and gives you the length. That was the feature I was proudest of. It sounds like nothing and is very hard to get right.
We wrote the tracking ourselves
And here comes the part where today I put my hand to my forehead and at the same time think: good that we did it.
We wrote the tracking ourselves. Not the whole chain, but the part that decides whether a scan holds up: how the points the LiDAR delivers and the movement of the phone become one coherent model. Where points belong. What is surface and what is noise. How you merge points you have seen twice, from two directions, with a slightly different position.
It was the hardest number crunching we had done up to then. Not machine learning, but geometry, numerics, memory. A model that still fits into a phone's memory after five minutes of scanning. A build-up fast enough that you can watch while walking. And all on the device, because we had decided that nothing leaves the phone.
No cloud, no backend, flight mode
That was the decision that determined everything: there is no cloud. Scanning works in flight mode. The model is built on the phone, saved, and only shared if you want, with whom you want.
In 2021 that was a selling point. All the other scanners uploaded the points and computed the model somewhere. We said: your bathroom stays on your phone. That landed us with all the number crunching, and it was right. Today, five years later, "on device" is again the argument everyone advertises with. We were early. Sometimes early is good. Sometimes it is just early.
Why, and why not further
Why did we do it? Honestly: because we could, and because the question after object recognition was on the table. We had a phone that recognises things and wanted it to understand the room. That is a reason. Whether it was a business model is another question.
The website shows prices: free, 6.99 a month, 66.99 a year. The 2022 factsheet says "Scan, Build, Collaborate", a desktop suite, a backend for collaboration with builders and electricians. That was the direction it was supposed to go: from the cool app to the tool with which someone on a construction site says, the socket goes here.
Why it did not arrive there I cannot say in one sentence today. Maybe too early. Maybe too much technology and too little customer. Maybe SKIMO came, the client projects came, life came in between. That is the hard thing about the why: you do not know it better afterwards either. You only see what was there.
What you would do differently today
Do not write the tracking yourself anymore. What we built back then exists ready-made today, on the device, in real time, better than ours. You take it and build what goes on top. Back then it did not exist. That is not a reproach to us, it is just the time that has passed.
Start with the customer, not with the scan. The electrician who wants to know where the socket goes was in the 2022 factsheet. He should have been at the beginning in 2020. Which decision should get better? The same question as everywhere.
On device stays. That was right and is more right today than back then. Whoever scans people's rooms sends them nowhere.
Live mode was the future, not the scan. Placing notes directly into the room without first building a model. In 2021 that was "the coolest mode", as the website put it. Today that is the mode everyone wants, with glasses or without.
And still, when I open the website and see the screenshots, the bathroom with the measurements, "222.16 cm", the note "this needs to be fixed": that was real. No simulation, no rendering, we wrote that on it on purpose. It ran. On a phone. Without the cloud. With tracking we wrote ourselves.
Sometimes you build something because the question is on the table, and you find the why only later, or never. Scan Your World was such a project. It was too early, it was too much technology, and it was a joy. All three are true.
Written by me. The thoughts, the values, the learnings, the mistakes: all mine. Grammar and spelling are corrected by our own twin model, trained on my texts. Sometimes a stumble stays in. That is mine too.