Solving MySQL / MariaDB startup / connection problems when running ZoneMinder in a Docker image

After a few years away from ZoneMinder, I decided to reinstall, but this time using a Docker image. I selected dlandon’s very helpful packaging rather than attempting to create my own image. I created the necessary directories (for the mapped in recording cache and for the MySQL database) and started up the image with the parameters as described in dlandon’s documentation.

However, the image kept refusing to start for me. I would get:

Starting MariaDB database server mysqld,
...fail!

First thing to do was to figure out what the problem was for mysqld. You can’t just run a shell inside the image though, as the image has quit by the time you’re looking at it. So let’s modify the image to start up with a bash shell (while keeping its state, so that we can check the logs, etc.) and poke around inside. I recommend storing all this in a script so you can repeat it! Here’s the script I used to commit a new copy with bash as the entry point, start it up, and then delete it afterwards (while it’s running you can exec bash in it from another terminal if you want to have multiple terminals poking around, which can be helpful):

YOURCONFIGPATH=your config path goes here
YOURCACHEPATH=your cache path goes here
YOURUSERNAME=your user name goes here

# Get the original container
CONTAINERID=`docker ps -a | grep zoneminder | awk '{print $1}'`

# Clone it
docker commit ${CONTAINERID} ${YOURUSERNAME}/test

# Start it with bash as the modified entry point
docker run -it --entrypoint=bash \
--name="Zoneminder2" \
--privileged="true" \
-p 8443:443/tcp \
-p 9000:9000/tcp \
-e TZ="America/Chicago" \
-e SHMEM="30%" \
-e PUID="99" \
-e PGID="100" \
-v "${YOURCONFIGPATH}":"/config":rw \
-v "${YOURCACHEPATH}":"/var/cache/zoneminder":rw \
${YOURUSERNAME}/test

# Clean up afterwards ...
docker images -a | grep "${YOURUSERNAME}/test" | awk '{print $3}' | xargs docker rmi -f
docker ps -a | grep "Zoneminder2" | awk '{print $1}' | xargs docker rm

Now you can poke around inside the container to figure out why mysqld isn’t happy running. You’ll find your key log file here:

/var/log/mysql/error.log

You may want to clean out the log file (it probably has some goop inherited from the original docker build) and re-run the service to get a clean and easy view:

rm /var/log/mysql/error.log
service mysql start

OK, so what you’re probably seeing in the log file is a warning about not being able to write a test file, and some following MariaDB errors about bad permissions (I’m no longer getting them, so this is from memory/Googling others):

[Warning] Can't create test file

This is indicating that MySQL can’t write to the directory you’ve pointed it to.

First, ensure that there are no basic permissions problems. I won’t get too into this, as you should be able to do this — but make sure that wherever you put the database (it’s going to be in ${CONFIG}/mysql) is fully accessible (e.g., the directory path can all be read, the directory itself is writable, etc.)

I’m assuming you have what look to be fine permissions — I sure did.

At this point, you probably have one of three culprits:

  1. If you located the CONFIG outside of the normal spots (let’s say, anywhere outside of /var), then it’s possible you’re running into MariaDB’s own protection. It’s got some settings to prevent writing to /usr, /etc, /home, and so forth. This wasn’t my problem, but I found it while trying to figure out what my problem was. If this is your problem, easiest would be to relocate the directory, but if you can’t, you may find some help in tinkering with the ProtectHome and ProtectSystem settings for MySQL in the configuration (inside the docker image! — which introduces its own long-term complexities).
  2. If you’re running SELinix in your host system, SELinux may be blocking access to your data directory. I’m not, so I can’t give you details, but you may find SELinux and MySQL by Jeremy Smyth to be of use.
  3. For me, the problem was that AppArmor (in my host system) was blocking access. I could see this very clearly (once I started looking) by checking /var/log/syslog and looking for apparmor lines. You can find the fix for this in AppArmor and MySQL by Jeremy Smyth . In brief, the fix is to add two lines to the bottom of /etc/apparmor.d/local/usr.sbin.mysqld (customized for your data location, of course):
/YOURDATAPATH/ r,
/YOURDATAPATH/** rwk,

Et voila, everything started working!

Firewall Port 9933 for My Singing Monsters

Just recording this, as I was only able to find an unconfirmed hint on a forum.

You need to open up port 9933 (not sure whether TCP or UDP … I opened both, and didn’t bother going back to experiment) on your firewall/router in order to enable the Big Blue Bubble iOS game My Singing Monsters to connect to its server. Otherwise, you’ll get the “Failed to Connect to Server” error from the game.

Hope this helps someone …

Fixing Facebook Chat (Messaging) on the iPad (iOS) by Opening port 8883 on the Firewall

My darling wife got a new iPad2 for the holidays.  Worked fine while we were out of the house … but as soon as we got home, the Facebook Chat (messaging) feature stopped working, and hung with a permanent “connecting”.  In case you’re not following, “chat” is what shows up in the right-hand column when you use Facebook for the iPad (latest version as of December 2011) in landscape mode (and only shows up in landscape mode).  To restate the problem (and help indexing <g>):  Facebook Chat (aka Facebook Messaging) on the iPad (right-most pane in landscape mode) hung with a “Connecting …” message due to firewall port blocking.

When I connected outside my firewall (directly to my DSL connection) it worked; it didn’t work from behind my firewall.  So … clearly a ports issue … but what ports?

Drove myself crazy trying to figure this out.  Nothing I could find by Googling.  Closest I could come was that Facebook Chat might be using XMPP (TCP ports 5222 and 5223), which makes sense … but opening those ports did nothing to solve the problem.

Eventually, I turned off the firewall (yikes) and turned on WireShark and ran the app.  Doing so, I saw a very strange pattern — connections to Facebook servers on port 8883, which is, technically, the “secure MQTT” port.  Long story short, I enabled TCP port 8883, and Facebook Chat suddenly started working.

Some additional notes:

  • I left XMPP (ports 5222 and 5223) enabled … so you might also need these ports, in addition to 8883.
  • 8883 is strange!  The only meaningful reference I could find was that SameTime, that old Lotus Notes IM app, used it.  Maybe someone at Facebook pilfered some old SameTime code?

If this helps you out, please leave a comment … I’m curious to know who else has encountered this, whether it’s something unique to me, or I just happened to stumble on it first!

Creating Large Partitions on Ubuntu

While I had my server open to mount the WD (formerly external) drive, I threw in the two 1.5Tb (!) drives I’d brought back from the US … yep, time to start regularly backing up all the machines sitting around the house.

GPartEd refused to create 1.5Tb partitions (kept getting the error “a partition cannot have a length of -1 sectors”).  Setting the disk label to GPT rather than MSDOS (as suggested here) didn’t do the trick.  Instead, what worked was a combination of cfdisk and mke2fs.  In my case (assuming /dev/sde):

sudo cfdisk /dev/sde
sudo mke2fs -j /dev/sde1

Finally, to get the volume ID (for ID-based mounting, which is always best!):

sudo vol_id /dev/sde1

… and add it to the fstab.