Jan. 12, 2012, 4:38 p.m.
—
Permalink
Queues, counters and polling How many times have you modelled a queue in SQL? What about event counters? Let's not even get started on polling for changes... Since we started using Redis we've noticed that most of our 'hard' modelling problems in SQL will fit into a Redis construct really ...
Read more
Sept. 24, 2011, 8:32 a.m.
—
Permalink
Spent a little while trying to find the size of a specific index this morning and couldn't find any documentation on how to do it. Eventually stumbled on it in db.collection_name.stats() > db.content.stats() { "ns" : "conversocial.content", "sharded" : false, "primary" : "main01", "ns" : "conversocial.content", "count" : 1924859, "size" ...
Read more
Sept. 21, 2011, 9:58 a.m.
—
Permalink
In a previous post I described Django SQL Sampler as a tool that helps you find the SQL queries that are consuming the most time on a production site. I've now renamed Django SQL Sampler to Django Sampler because it now does much more. It now has a plugin architecture ...
Read more
Sept. 12, 2011, 7:32 a.m.
—
Permalink
An interview is an opportunity for you to tell me about your your legendary hacking and awesome communication skills. A contribution to an open source project shows me this, and more. Designers don't turn up to interviews with just their CV, they bring along a portfolio of their work. They ...
Read more
Sept. 4, 2011, 7:18 p.m.
—
Permalink
Any site of a large size with have lots of different queries going on. If you're having performance trouble it's easy to find and fix the ones that are causing trouble. It's not so easy if you have a query that runs moderately fast but is run a lot. It's ...
Read more
Aug. 26, 2011, 4:36 p.m.
—
Permalink
I wanted to profile our system that is using eventlet. I wasn't happy with any of the existing solutions so I rolled my own: https://github.com/colinhowe/eventlet_profiler. The existing solutions have a few failures: You can't figure out how much time a specific function spends calling another function They generally use CPU ...
Read more
Aug. 24, 2011, 6:39 p.m.
—
Permalink
Whenever I'm pair-programming with someone we often hit a spot where we both want to experiment and try stuff out - different ways of approaching a problem, playing with APIs, whatever. Going back to your own computer when you hit this spot isn't great. By moving away from the other ...
Read more
Aug. 22, 2011, 5:10 p.m.
—
Permalink
In a previous post I looked at the performance hit you get in MongoDB when the working set is larger than the memory available. I thought I'd have a look to see how MySQL fares under similar conditions. The experiment was very similar experiment to the experiment for MongoDB. Setup ...
Read more
Aug. 17, 2011, 9:46 a.m.
—
Permalink
When using eventlet you typically have a lot of concurrent things all being processed at once. Trying to identify problems in highly concurrent systems can be hard. To solve a problem we had I created eventlet visualiser (on github). This allows you to see the life span of all your ...
Read more
Aug. 9, 2011, 8:17 a.m.
—
Permalink
I sometimes need to do a search and replace across lots of files. So, I made a function in my .bashrc to make it easy: function find_replace { FIND=$1 REPLACE=$2 echo "Finding $FIND and replacing with $REPLACE" grep -rl $FIND . grep -rl "$FIND" . | xargs sed -i -e ...
Read more
