Notes on the httprouter Golang library

For most simple to moderately complex web servers in Golang, I have always preferred to use the standard net/http library, laboring through the parsing of RequestPaths and query parameters when necessary.

But when the boilerplate code for request parsing starts obscuring the business logic, I tend to look out at other libraries. I however have always hesitated using too heavy a framework to keep dependencies simple.

For HTTP routing, for example, I have found httprouter to satisfy most of the gaps in the standard library that I have frequently faced, and at the same time the library is totally dep free!

Here are some really quick notes about using the library, mostly for my own reference. :)

Automating MySQL GTID Migration With Ansible

MySQL 5.6 onwards introduced GTID based replication drastically simplifying replication setup and increasing reliability. New MySQL cluster 5.6+ setups are already done with GTID enabled by default.

But if you are one of those people who migrated from a pre-5.6 MySQL version, you probably avoided enabling GTID to make the upgrade easier. In such a situation, typically, you would upgrade the slaves to the new MySQL version, and then failover the master to an upgraded slave. This would let you do an online upgrade of a MySQL cluster from a pre-5.6 to a 5.6+ installation, but you will not be able to do this with GTID enabled.

I recently had a requirement to move a bunch of such MySQL 5.7 clusters using the old binlog position based replication to a GTID based replication setup.

Now there is a pretty good official document about how to do this manually, which you should definitely read up before doing this. But it is a pain to do this manually on a bunch of servers.

I captured all the steps mentioned in the document into an Ansible playbook to automate the whole process. It also includes a procedure missing in the official document, to actually flip the ongoing replication to GTID protocol.

The playbook source is here with step by step documentation here.

Creating a Certificate Authority in 2020 for Your Soho

I have a couple of systems at home which provide web services, like my Intel NUC and my Synology NAS, and I have been wanting for a while to move all of them to a proper https only environment.

But my biggest hurdle for doing that, has been the enormous pain in managing certificates in a way that makes everybody - the servers, the browsers, the local http clients happy. From my previous attempts, there was always the browsers which annoyed me to no end, and I ended up getting by using improperly made self-signed certificates and accepting all the invalid certificate warnings that my browser threw up.

So this Friday night, I spent my late night hours trying to get at the bottom of it all, and several frustrating hours later, finally made everybody happy.

Open Every Link in a Web Page In a New Tab

Found this nifty trick in a github comment.

If you add this bit of code in the <head> section of an HTML 5 page, clicking on every link will open in a new tab.

<base target="_blank">

This tag is mostly used to set a URL to resolve all relative links on a page. But it offers an additional attribute target to set a default target policy for links.

You can, of course, override the target attribute on a link to link basis, if you are using this feature.

Ansible privilege escalation with expect when you don't have root shell privileges

The default Ansible privilege escalation mechanism requires broad sudo privileges. If your production environment gives you sudo access but bars you from getting a root shell, you are out of luck. As, the doc says - you cannot expect Ansible to work when sudo commands are restricted.

Privilege escalation permissions have to be general. Ansible does not always use a specific command to do something but runs modules (code) from a temporary file name which changes every time. If you have /sbin/service or /bin/chmod as the allowed commands this will fail with ansible as those paths won’t match with the temporary file that ansible creates to run the module.

Integer maths in Go using constants with exponential notation

I seem to learn more about the nuances of the Go language every other day. Sometime back, I had looked at how Go untyped constants work during maths operations with typed variables. I just found another significant part of the spec that I had previously glossed over, this one is also about untyped constants - numeric constants in Go live in an unified space with arbitrary precision and a fungible numeric type.

Runit, Chpst and ulimit defaults

So I ran into this problem at work today with an runit based service breaching open files limit.

My first thought was to increase the system ulimit for nofile in /etc/security/limits.conf. I changed this from 30k to about 60k. But strangely, the service still keep dying.

Using Zap - Working With Global Loggers

Sometimes instead of creating a logger and then passing it around, it is convenient to just use a global logger.

The standard log library allows you to both create a custom logger using log.New() or directly use a standard logger instance by calling the package helper functions log.Printf() and the like.

zap provides such a functionality as well using zap.L() and zap.S(), however using them didn’t seem so straight forward to me.

Using Zap - Creating custom encoders

The various implementations of field encoders provided in zap can sometimes feel inadequate. For example, you might want the logging output to be similar to that in syslog or other common log formats. You might want the timestamps in the log to ignore seconds, or the log level to be wrapped within square brackets.

To have your own custom formatters for the metadata fields you need to write custom encoders.

Using Zap - Creating custom loggers

Using the logger presets in zap can be a huge time saver, but if you really need to tweak the logger, you need to explore ways to create custom loggers. zap provides an easy way to create custom loggers using a configuration struct. You can either create the logger configuration using a JSON object (possibly kept in a file next to your other app config files), or you can statically configure it using the native zap.Config struct, which we will explore here.