The Hybrid solution

Yesterday I read on Shawn Blanc's excellent blog about his hybrid solution to task management. Like me, he enjoys using a notebook for task management, but also sees the big advantages of having a digital system (Shawn is a big Omnifocus user). Shawn came up with a hybrid system to combine the two, which seems really smart, so I decided to try it out.

Like Shawn, I was using a digital task management tool, Things. I was also using the Bullet Journal system for my notebook. Combining the two was difficult for me and I could not find a system that worked naturally. For a few days now I have been using Shawn's system and it is working really well. Not only am I getting more things done per day, but I also feel more calmed and less overwhelmed by everything that needs to get done each day.


Django ORM group by

Filtering in Django over foreign keys or many to many relations's values has, as a buy product, the possibility of creating duplicates of the object you are actually filtering from. A simple way to avoid this is to group by on the primary key of the model. Django, however, does not have group by in the ORM. There is however a way to make Django add it. By slightly abusing annotation it is possible to add just such a claus.

from django.db.models import Count obj1.objects.filter(obj2__value=1).annotate(Count('pk'))

This will add the group by on the obj1 primary key and avoid duplication on the result set.

Monumental Valley

Monumental Valey normally speaking I don't really like gaming on the iPad. call me old fashioned but I like a controller with some physical buttons on it. To me even PC gaming is somewhat off because of the keyboard and mouse1. That said there are games that fit the platform so good that I can fully enjoy them without filling handicapped by the lack of a controller. Monumental Valley is such a game. Inspired by Asher drawings the game utilizes impossible geometrics to make some interesting puzzles. You are guiding Ida, a princess through the impossible geometric world manipulating it to make paths for her to walk. The graphics are stylized and beautiful. The game is challenging, if somewhat short (It took me a little over an hour to finish). if you enjoy puzzles and some dimensional thinking you should definitely pick this one up.

  1. I know it is better in absolute terms. I just prefer a controller. 

Threading local and Django

It is rare for me to find something in Python that does not work as I expect it to. Generally speaking, the way I think seems to match the way Python does things. Thats a great advantage to have when working with the language. However when Python does not act as expected it is hard for to find out what is actually going on, because I have to try and think in ways alien to me. Today I met just such a issue within threading.

I am currently working on a project which involves a lot of aggregation of data. Recently the decision was make the application international. Part of the process was creating international databases and split the aggregated data across these databases. Most examples of using multiple databases use either the app, or the specific model to determine which database to use. My use case was more difficult as it involved selecting the database based on the actual data. If the data source was, for instance, from Spain, the spanish database should be used. The data was streaming in all mangled up so it was not possible to use the source as a way to distinguish the data. Furthermore, the data was not coming through a request (where middleware can be used) but synced in from a remote database via a cron job that the proceeds to aggregate the data. To set up the database routing I created a router class. The problem is, there is no way to actually pass data to the routing class. At the moment the hints parameter contains only the instance, if it exists. So for a newly created model object it's empty. My idea was to try and use the threading.local() to communicate between the aggregation function and the db routing class.

Why I'm not using using

First however let me explain why I have not chose to use using. The problem is that when aggregating, the application creates a few different objects, depending on the data. These model objects are created through proxy analyser classes. Using the manual method would not only involve a lot of code, it will also make debugging difficult.

What didn't work

My first instinct was to thing that in each file I needed To access the data I just needed to add the following lines:

import threading
local_storage = threading.local()

And that local storage would be consistent across the whole thread. Unfortunately, it either wasn't, or the code was running on two separate threads, which I don't think it did. This might be a good time for a disclaimer. I am by no means a threading expert. What I say may be really off base. All I know is what I observed and what did, and did not work for me. I added this code to both the db routing class and the aggregator. The aggregator would add an attribute to local_storage and the db routing would check for it to determine the routing. This attempt failed. After some debugging I found that the object created in the aggregator and the one created at the db router were, in fact, not the same. I figured if I define it one of the two, and import it for the other then surely this would be the same object. It was not. I think it probably has something to do with how import actually work, but I'm not sure. I was starting to get frustrated. Googling around didn't really turn up anything significant. I was all but ready to give up on threading.local when I found the django-tools Threadlocal middleware on github. It was using the mechanism I had in mind to make the request available everywhere. I was quite sure that this code worked (because everything you read on the internet is true, right?). So what was I doing wrong?

What did work

The difference seemed to be in what was actually imported. The middleware was defining the local storage and to access the local storage the middleware module was imported and the local storage is then accessed via functions in the module. I did not really see the difference but figured it was worth a shot. I added a new module that defined the local storage and added getters and setters. To my surprise it actually worked. I have no idea why this method worked while the other failed. I am guessing, as I said, that it has something to do with how threading and import work, and what is passed by reference and what is passed by value. One day I will have to dig deeper into this but for now this will do


The threading.local() object offers a thread safe manner to pass data between different parts of the django application when normal parameter passing is not possible. For it to work properly you need to create a proxy module with a getter and setter (and a deleter) and then import that module to each module that needs access.

That First Spring Weekend

That first weekend , when its sunny and just barely warm enough for shorts is very special here in the Netherlands. Everybody will go out dressed way to optimistically, the terraces will be flooded with people and the atmosphere is very jolly. Last weekend was suck a weekend. As a mountain biker, its also a great opportunity to do some riding.

This year we were lucky in that it has been dry for a few days leading to this weekend. The trails were proper dry by the time weekend was upon us.

Saturday I did not have a lot of time so I managed a short ride on my fixie. Even though it was short it was very relaxing. There is something to be said for the simplicity and focus of riding fixed that really helps clear the mind. I checked later and it was exactly a month ago that I last sat on a bike saddle.

Sunday the weather was even better and I had more time on my hands and so I rolled to Amerongen. The last few days have been dried and so I expected the trail there to be in good condition. I was not disappointed. There was very little mud and I could push hard. It was months since the last time I rode my mountain bike so I considered taking it easy. For the first 20 seconds. Then I put the hammer down.

The last few months have been incredibly busy (home renovations, a new baby). Training had to take a back seat. Having trained very little the last few months, I was curious as to how much my fitness has regressed. The only way to find out was to push as hard as I could and see what happend. I hammered through Amerongen, pushing as hard as I could. By the time I was up the Amerongseberg I was feeling low on energy. I ate half a bar, took a few deep breaths and continued to Leersum. I started Leersum pushing hard. I could feel power returning to my legs. These new duursport bars were proving their worth. The Leersum trail flew by and in 20 minutes I was finished and ready for round 2 of Amerongen. By now my legs were in constant, but manageable, pain. Munching on the second half of the bar, I started my second run of Amerongen.

About half way through, I started getting cramps. It was time to go over to chill mode. Pushing really hard voor over an hour without warming up (I know I know, bad idea), and without some training buildup was taking its toll on my legs. I finished the trail run in chill mode, crawling up the last climb.

Only when I got back to the car and got of the bike did I notice I was grinding like an idiot. It think the grind took hours to disappear. It felt so good to be back on a trail. When I got home I looked at my data. These runs were my 3rd fastest ever runs on both Amerongen en Leersum. Not bad. I even managed to ride a few PRs, which is a great boost to motivation. I can't wait to get back.

What problem

Great short piece on minimal mac. I to noticed that not following this makes me spend attention, and money on unneeded, unimportant technology with no real gain.

Week Lessons 3

Mercurial Bookmarks

If one bookmaker is a direct decedent of another bookmaker, it is not possible to use hg merge between the two, since mercurial does not have a fast-forward option like git. The solution is to simply, update. Do note that it seems impossible to update to a bookmark that is half way through the tree. There is great stack overflow question and answer about this.


page gave a 500 error logs said SuspiciousOperation error. access was denied to file

problem was css compressor if more then 1 file was compressed. Reason was a bad media_root setting

Mixed content

iFrame https in a http parent does not qualify but this is good to know - Mozilla blog post

Check in js if in iFrame

Window top === window self

Django timestamp field

See code

Week Lessons 2

Use curl to post data

While developing an api, I needed to do some posts to a certain url. The fastest way I could think of was to use the command line curl command. However, by default curl does a GET request, and of course, sends no data. However a short googling trip took me to this superuser answer, which demonstrates how to do it. I was posting json to my api endpoint in no time. Example:

curl -X POST -d @filename.json --header "Content-Type:application/json"

Which will do a post using data from filename.json to the given url. The request header content type is also set to json. Of course if your not posting json this should be changed to reflect the data.

Arik Einsten

Arik Einstein Yesterday, at the age of 74, Arik Einstein passed away. Today he was buried in Tel Aviv, his and mine beloved city. Mr Einstein has been so influential to israeli culture that it is hard to explain it to non Israelis. There is simply no Arik Einstein here in the Netherlands. He was a musician, a singer, an actor, a comedian. He was enormously influential because he was simply incredibly good at what he did. It sometimes seemed like success was just an unintended side effect. He never chanced it, he was just to good to not achieve it.

Writers much better then myself will no doubt write about him in the coming weeks, so I will focus on my personal angle. In august of 2000 I moved to the Netherlands. I moved alone and knew no one here. The first few years were at time exciting, at time magical and at times very lonely. It was Mr. Einstein's "In San Francisco On The Waterfront" which have became the hymn of my life at those years. We never actually met, and yet there were time in which I would sit and listen to that magical song and it felt like he, and only he fully understood what my life was like. That sort of bond feeling never left me. The first hebrew song I sag to my son was Mr. Einstein's "Saturday Morning".

All I have left to say is thank you, Mr. Einstein. Thank you for everything. You will be dearly, dearly missed