Linux notes

Last month we had a very cool challenge at SUSE: Replace any proprietary application with an Open Source replacement, and SUSE will donate to Open Source organizations. My only problem: I didn’t have any applications left which I could have migrated ;-) But then a tragedy happened…

First of all I should write: Valetudo’s documentation is excellent, it doesn’t really need any further explanation! So I’ll just write about the process itself and my thoughts about the new robot - most of all because my company internal post announcing the migration got quite some interest - and I was asked to write a blog post about my experiences. So here we are!

The Tragedy

My old Roomba 760 was a reliable partner for around 14 years now, but recently it got stuck beneath the shoe shelf so unluckily that it ripped of its sensor for finding home.

Also other components started to get more and more unreliable, so it really seemed to be the time to finally look for something new. The Roomba was a model which was driving around semi-randomly (with some strategies like driving along a wall), but it managed to reliably clean whole flat, consisting of 5 rooms.

The Requirements

Of course I wanted to have an adequate replacement, meeting the following criteria:

• No app required
• Should also be able to mop the floor
• Possibility to only clean certain rooms (e.g. to mop the bathroom more often than everything else)

Easy, right?

Well, 0 robots fulfilled these criteria by default! Either you get something driving around randomly again (the cheap models), or you have to connect to the vendor’s cloud via an app (everything else). Or in other words: If you want something “intelligent”, a cloud connection and an app are required. That’s not acceptable for me, and this is where Valetudo comes in!

Choosing a robot

To install Valetudo it must be possible to gain root privileges in some way.

The biggest catch: At the time of this writing the newest supported robot is already over two years old, and even the newest supported models are getting increasingly hard to find in new condition. Otherwise used models are of course also a option and considerably cheaper.

I wanted something new, and in the end I decided for the dreame X40 Ultra Complete because of it’s (theoretical) option to connect the fresh and used water tanks directly to the water supply. I only later noticed that this extension is sold out already… Another feature I wanted to have is the extensible brush - but see below whether it’s really worth it. The alternative for a new robot are the Eureka ones which seem to have a significantly better brush by default which is cutting hair before it can wind around the brush. Another advantage of the Eureka robots: you don’t need a breakout board, you can connect a USB cable directly.

The quality of the robot seems to be (surprisingly!) good to me - but of course I don’t have any long term experience yet.

Operation without a cloud or app connection

One important finding: You can use these robots without a cloud by default! My robot has three physical buttons, one of them to start a cleaning / mopping run.

On the first run it will map the environment (while cleaning already) and store it locally - subsequent runs will be significantly faster then.

So far so good, but unfortunately this way you don’t have any fine control: You can’t edit the generated map, you can’t set any timers, you can’t chose to only use the vacuum and you can’t adjust the water or fan intensity. But in general it works :-) This will most likely work for all the other supported robots, too.

What does Valetudo do?

First of all (and probably a bit too simplified to get the point across): Valetudo is just a web service on your robot which will provide access to the features of the robot from your local network. It will not replace any of the robot’s firmware for the components: Cleaning or mapping won’t get any better (or worse) than with the original system.

In the end you will still have the same robot you had, but you will now have root access to it, and you will have a webserver running to control it from you local network without any proprietary Cloud connection.

Installing Valetudo

So let’s finally install Valetudo to get access to all the features!

As mentioned already: The documentation is excellent, just follow it word by word. The instructions will greatly differ according to the model you chose! Some robots will require full disassembly, others just need a USB cable and an ADB connection - so you should read the instructions before you buy any robot and see whether you are willing to do what is required.

In general the process consists of three steps:

• Connect the robot and gather some required information
• Root your robot
• Install Valetudo

For the dreame robots you will need a Breakout adapter (yes, you probably really want one - finding non-standard 2mm pins to attach everything manually into the pinout connector is even more difficult than to find someone with a Breakout adapter :-)): It will map the pins on the robot to a Micro USB port. Don’t be shy and ask around: At work or in a nearby hackerspace - it’s very likely someone near you has one already. Otherwise you would have to manufacture the board yourself - or buy another robots which doesn’t need a Breakout adapter.

Once I found a friendly colleague with a Breakout adapter the installation was totally flawless for me - except for two stupid mistakes which cost me some time to find out:

• When the breakout board is connected to the dreame and you turn it on, the button LEDs will start blinking. If everything is correct they will continue blinking. If the blinking stops after a while and you hear the startup melody, then the robot just booted normally. In my case one of the connectors of the breakout board wasn’t soldered correctly, so it never got into fastboot mode.
• If you need a USB cable: Make sure that it got it’s data wires connected! Some cables are only meant to be used for charging and don’t have the two middle wires connected. In the case of the dreame robots: if you connect the USB cable to your PC and it doesn’t show a new device, use a USB known to transfer data.

Which robot is better?

So now the final question: Was the new robot really an upgrade?

Well, unfortunately the answer is not that easy.

Positive

• The mopping feature is amazing: The wooden floor is really shiny again, and the included cleaning ingredient seems to work very well.
• The mapping feature is really cool, and rooms are detected very reliably

Negative

• The vacuum doesn’t work very well. Even though it has an extensible sidebrush it doesn’t use it on carpets at all, and on borders and objects such as chair legs it’s not cleaning well, leaving whole areas uncovered.
• “Everything is a wall”: When the sensors detect any obstacle, the robot won’t go there, even if it’s just a curtain or sofa cover. These areas can be tagged accordingly, but it’s still not very reliable…

It’s funny, but the old chaos principle of the Roomba resulted in better vacuum cleaning results than the highly sophisticated systematical approach of the dreame. The mopping functionality is really worth it though. Due to that I have to say: You probably don’t have to spend that much money to get the latest and greatest features. An older model without the extensible sidebrush will probably clean exactly the same :-)

Alex the Allegator 2 also didn’t have a Linux port yet :-)

This time it’s a puzzle game.

Dependencies

• Allegro 4.x

Downloads:

• Source code
• Latest binaries for Linux x86_64

I recently stumbled over some old Linux games by Free Lunch Design (Alex the Allegator 4 to be exact) and found out even more had the source code available!

This is a quick DOS to Linux port of the original Alex the Allegator 1 from the Alex the Allegator series, which doesn’t seem to have been ported yet.

I admit the game is very minimal - my first playthrough lasted about 3 minutes - but now I (and hopefully also you) know what we missed at least ;-)

Dependencies

• Allegro 4.x

Downloads:

• Source code
• Latest binaries for Linux x86_64

Have fun!

From time to time I stumble about older applications (usually games) which use MIDI for music playback. These are some notes on how to get things working with various libraries and will be extended from now on every time I stumble upon something.

General

MIDI was originally meant to be supported by hardware, but when we are talking about today’s soundcards you won’t find any consumer sound card with hardware MIDI support any more. Because of that you need software to do the job.

This is where software such as TiMidity++ or FluidSynth come into play, but this article will only cover the requirements for specific libraries. The focus of this article are the requirements of specific applications / libraries however; for a general overview see e.g. the ArchWiki entry on MIDI.

The most important thing first

If everything you try just doesn’t want to work, make sure that the applications and libraries you are using are compiled with MIDI support built-in! Nothing is more frustrating than debugging for hours just to find out that the packager didn’t care about MIDI support when creating the package. Yes, that happened to me several times, so if in doubt, do yourself a favor and compile the relevant parts yourself just to make sure.

Allegro 4

Allegro is a cross-platform library for creating multimedia applications, including games. Up to version 4 it also included MIDI support.

On Linux the following MIDI drivers are supported:

MIDI_OSS - Open Sound System
MIDI_DIGMID - sample-based software wavetable player
MIDI_ALSA - ALSA RawMIDI driver

I never tested OSS MIDI setup.
Allegro’s ALSA driver unfortunately only supports raw MIDI devices (which could be emulated by TiMidity++ if one wanted).
If you don’t have one set up, the autodetection will use the DIGMID software emulation, and that’s the easiest to set up. It just requires one file with the MIDI instruments in a specific location - otherwise you simply won’t hear anything.

I’ll quote the readme.txt file:

The DIGMID wavetable driver uses standard GUS format .pat files, and you
will need a collection of such instruments before you can use it. This
can either be in the standard GUS format (a set of .pat files and a
default.cfg index), or a patches.dat file as produced by the pat2dat
utility. You can also use pat2dat to convert AWE32 SoundFont banks into
the patches.dat format […].

HOWTO

• Get a sound bank in “SoundFont 2” or “GUS” format (see link to ArchWiki above for links).
• If it’s in “SoundFont 2” format: Convert it using pat2dat -o patches.dat /usr/share/soundfonts/CT8MGM.SF2 (I’ve used the SoundFonts included with my ancient SoundBlaster 16 for this).
• Copy the file(s) to /usr/share/allegro/ (which I’d recommend, then all Allegro based applications will use it) or directly into the application’s directory.

Or: How to extract “Resource Forks” on Linux

Or: What happens if you want to access Mac data from a game disc without a Mac? A self experiment.

Intro

You may know the Marathon series: Three first person shooters games, which may be seen as a predecessor of the later “Halo” series by the same company. There is one caveat though: Back then, developer Bungie Software was a very Mac centric company, and most of their games didn’t even have ports to Windows or other platforms.

The exception in the Marathon series is the second part, “Marathon 2: Durandal”, which is the only one that was also ported to Windows. But not just that, the game was even localized into German (and probably a few other languages as well).

Fast forward: In 2000, something wonderful happened. The source code of the game engine was released as Open Source, and in 2005 the games themselves were released as freeware. That is: The Mac version of the game engine was open-sourced, and the Mac version of the games were released as freeware. And just the English versions of them.

So now I’m looking at my beautiful German box of Marathon 2 and am wondering: Can I also play that German version on Linux?

Of course that’s possible, but it turned out that getting the data files is more complicated than expected…

Mounting the Mac part

So let’s have a look at the disc: It contains both an ISO9660 part for the Windows version and a HFS part for the Mac version.

The source code, meanwhile curated by the wonderful people of the Aleph One project, is - of course - only compatible with the Mac game data, so let’s try to mount the HFS part of the disc (by default the ISO9660 part would be mounted):

mount /dev/sr0 -t hfs /media/cdrom

If that command fails for you, your distribution probably didn’t include the hfs kernel module - but don’t worry, you won’t need it anyway. Apart from some strange characters (the kernel hfs module doesn’t seem to be very good with special characters - you can see the output in the “HFS” link above) all files seem to be there.

So: Let’s try to start the game:

$ alephone /media/cdrom/Marathon 2 �/
[…]
fatal alert (ID=-1): Please be sure the files ’Map’, ’Shapes’, ’Images’ and ’Sounds’ are correctly installed and try again. (csalerts_sdl.cpp:202)

OK, the manual mentions that you can start the Mac version directly from CD, so something seems to be off.

$ ls -l /media/cdrom/Marathon 2 �/
total 33648
-rw-r–r– 1 root root        0 Nov  6  1995 Iconr
-rw-r–r– 1 root root        0 Oct 10  1996 Images
-rw-r–r– 1 root root        0 Nov  6  1995 Manual (english)
-rw-r–r– 1 root root  5704325 Oct 14  1996 Map
-rw-r–r– 1 root root   407802 Oct 21  1996 Marathon 2
-rw-r–r– 1 root root  4136840 Jul 13  1995 Music
drwxr-xr-x 1 root root        4 Dec 10  1996 Physics Models/
-rw-r–r– 1 root root 10016800 Nov  6  1995 Shapes
-rw-r–r– 1 root root 14183536 Nov  6  1995 Sounds

What’s that? The Images file is empty? What happened here?

(One may wonder: How did I enter the unprintable character if tab completion stops at exactly that place because of another directory starting with the same name? cd /media/cdrom/Marathon 2 [^S], followed by pressing tab. I know zsh users will probably laugh now because they can simply cycle through the directories with tab by default…)

Exporting Resource Forks

It was clear now that either the kernel driver had a bug here, or that the information would be stored elsewhere. To exclude a kernel driver bug I decided to cross-check with another open source tool: HFSExplorer also promises to be able to view HFS file systems.

And surprise: HFSExplorer is able to decode the special file name characters, so we now know that the game is stored in folder called “Marathon 2 ƒ”, but the Images file is also shown with a size of 0 byte here.

The HFSExplorer download however does also include a command line utility called unhfs, and Heureka! The following command revealed the mystery:

unhfs -resforks APPLEDOUBLE /dev/sr0

Now there are suddenly two files: Images and ._Images - with a size of 4,8M that seems to be the file we are looking for. file ._Images reveals that this is a AppleDouble encoded Macintosh file. So: what the heck is this again?

Decoding AppleDouble encoded Macintosh files

For a short description of what the AppleDouble format is see e.g. the AppleDouble documentation of the Archive Team. In short: It’s a Metadata store - basically some parts of the data is stored as metadata, while other parts are stored in the actual file.

Unfortunately HFSExplorer can only export the metadata in AppleDouble format, while Aleph One can only read AppleSingle, MacBinary and RAW data. One option may be to convert the data to AppleSingle or MacBinary, but I haven’t found a tool which would create working files. So let’s create RAW files instead.

Analyzing the metadata is possible with several tools: apple_dump (from the Netatalk project / package will print some information, including an offset of 38 and a raw dump in hex and ASCII.

$ apple_dump Images | less
“Images” is not AppleSingle/AppleDouble format.
“/xxx/Marathon 2 ƒ/._Images” is found.
Dumping ”/xxx/Marathon 2 ƒ/._Images”…
——————————————————————————-
MagicNumber: 00051607                                        : AppleDouble
Version    : 00020000                                        : Version 2
Filler     : 4D 61 63 20 4F 53 20 58 20 20 20 20 20 20 20 20 : Mac OS X        
Num. of ent: 0001                                            : 1

——————————————————————————-
Entry ID   : 00000002 : Resource Fork
Offset     : 00000026 : 38 
Length     : 004C21F8 : 4989432

-RAW DUMP–:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F : (ASCII)
00000000   : 00 00 01 00 00 4C 20 F2 00 4C 1F F2 00 00 01 06 : …..L ..L……
00000010   : 00 00 23 4C 0B 74 72 6F 6F 70 65 72 2E 72 61 77 : ..#L.trooper.raw
00000020   : 02 00 00 00 72 73 72 63 52 53 45 44 01 00 FF FF : ….rsrcRSED….
00000030   : 06 49 6D 61 67 65 73 49 02 00 00 00 69 6D 67 32 : .ImagesI….img2
00000040   : 35 32 2E 34 01 00 00 35 00 2B 00 00 00 00 1E A7 : 52.4…5.+……
00000050   : 00 00 69 6D 67 32 35 32 2E 34 01 00 00 35 00 2B : ..img252.4…5.+
00000060   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : …………….
00000070   : 00 02 AC 28 59 B0 00 00 00 00 00 4C 21 F8 00 00 : …(Y……L!…
00000080   : 00 00 00 00 00 00 1B 00 00 00 23 4C 15 77 65 61 : ……….#L.wea
00000090   : 70 6F 6E 73 2D 69 6E 2D 68 61 6E 64 2E 70 6C 61 : pons-in-hand.pla
000000A0   : 74 65 02 00 00 00 50 49 43 54 38 42 49 4D 05 00 : te….PICT8BIM..
[…]

Despite of its name I ironically couldn’t find a way to just dump the data with apple_dump instead of printing the nicely formatted output.

Similarly Lsar from The Unarchiver (a collection of unpackers for various typical Macintosh formats) will also print the offset:

$ lsar -L ._Images
._Images: AppleSingle
._Images: 
  Name:                      ._Images
  Size:                      4.99 MB (4.989.432 bytes)
  Compressed size:           4.99 MB (4.989.432 bytes)
  Is a Mac OS resource fork: Yes
  Index in file:             0
  Start of data:             38
  Length of data:            4989432

But also unar, the extraction part, didn’t want to dump the actual data.

But if it’s really just an offset of 38: dd to the rescue ;-)

dd if=._Images of=Images.rsrc bs=38 skip=1

In case you are wondering about the extension: .rsrc is used by Aleph One to look for the RAW resource data.

Repeat that for the Map file:

dd if=._Map of=Map.rsrc bs=38 skip=1

And voilà: Alph One will now accept the data files, and most components, including the main menu and the ingame texts are localized now! Just the UI itself seems to be hardcoded.