&lt;ARCHIVE&gt; Wilmer's stuff https://wilmer.gaa.st/blog/ So today I... en Serendipity 2.0.3 - http://www.s9y.org/ Tue, 06 Oct 2020 09:42:00 GMT https://wilmer.gaa.st/blog/templates/default/img/s9y_banner_small.png RSS: &lt;ARCHIVE&gt; Wilmer's stuff - So today I... https://wilmer.gaa.st/blog/ 100 21 Dear /dev/null, II https://wilmer.gaa.st/blog/archives/99-Dear-devnull,-II.html Intarweb Life https://wilmer.gaa.st/blog/archives/99-Dear-devnull,-II.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=99 2 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=99 nospam@example.com (Wilmer van der Gaast) <b>tl;dr:</b> 302 → Pelican (URL not yet known)<br /> <br /> It's time to simplify my web presence. This massive ball of 4yo PHP with an unknown number of security risks should go, given how "much" I've been posting over the years. (Heh is anyone even reading this? I'll never know because I've already CSS-disabled the comments form. :-D)<br /> <br /> I do like the idea of Pelican for static blogging, for dumping some content with more depth than Twitter every now and then. Sadly I couldn't figure out how to have it import my S9Y posts, so let's instead resort to recursive wget. Added benefit: Preserves all URLs, no dead links \o/ Just in case some of this may actually still be useful some day.<br /> <br /> ... if high quality blogs and RSS readers were still a thing, that would be nice though. Maybe one day I'll go expore..<br /> <br /> <b>PS:</b> On the topic of quality: The "Preview" button in S9Y appears to be the Publish button or something, and I ended up with 7 drafts of this post published in random order. O_o Also, the delete button isn't actually working.<br /> Sun, 04 Oct 2020 23:02:00 +0000 https://wilmer.gaa.st/blog/archives/99-guid.html Puppet 4 on Debian 9 Stretch with nginx (with Puppet 3 agents if needed) https://wilmer.gaa.st/blog/archives/78-Puppet-4-on-Debian-9-Stretch-with-nginx-with-Puppet-3-agents-if-needed.html Linux OSS https://wilmer.gaa.st/blog/archives/78-Puppet-4-on-Debian-9-Stretch-with-nginx-with-Puppet-3-agents-if-needed.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=78 0 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=78 nospam@example.com (Wilmer van der Gaast) As part of upgrading my machines from Jessie to Stretch, I finally had to pick up Puppet 4. My hosts running testing were trying to do so for a while already, but since Puppet requires the master to be newer than the agents, I've always just had a pinning rule in place to stick all machines to Puppet 3.x.<br /> <br /> It's been quite the operation and I'm not done yet, but let me write down some of my findings for others to maybe use. As always, there are many different ways to achieve this goal, there are existing docs, but they're all outdated in one way or another. (As surely this one will be in a year.)<br /> <br /> <br /> During the upgrade, one of the first issues I ran into was that Puppet 3 agents are simply not compatible with Puppet 4 masters. When you try to have them connect, you'll get a poorly formatted error message that looks more or less like:<br /> <br /> <pre>Warning: Find /production/node/.... resulted in 404 with the message: {"message":"Not Found: Error: Invalid URL - Puppet expects requests that conform to the /puppet and /puppet-ca APIs.\n\nNote that Puppet 3 agents aren't compatible with this version; if you're running Puppet 3, you must either upgrade your agents to match the server or point them to a server running Puppet 3.\n\nMaster Info:\n Puppet version: 4.8.2\n Supported /puppet API versions: v3\n Supported /puppet-ca API versions: v1","issue_kind":"HANDLER_NOT_FOUND"}</pre>As helpfully pointed out by someone on <a href="https://tickets.puppetlabs.com/si/jira.issueviews:issue-html/SERVER-300/SERVER-300.html">this ticket</a>, this is actually just a generic 404 message that kind of assumes you're the wrong Puppet version talking to this master. Now the Debian NEWS file tells you that for Puppet 3 agents to work, you need to switch from the old stand-alone webrick master to Puppet master running under Passenger (kinda Ruby's equivalent of Python's WSGI?) One important note here is: You won't find this information in official Puppet docs, because this compatibility is actually a Debian-specific patch!<br /> <br /> (And another fun note: <a href="https://docs.puppet.com/puppet/5.1/passenger.html">Puppet Master inside Passenger, too, is deprecated in v5</a>, which kind of summarises my whole experience with Puppet TBH.)<br /> <br /> Anyway, for getting Puppet+passenger to work, there's the <a href="https://packages.debian.org/puppet-master-passenger">puppet-master-passenger</a> package. Which I didn't want to use though, because I've worked reasonably hard about a year ago to migrate away from Apache.<br /> <br /> <b>Screw backward compatibility, just give me Puppet+nginx+Passenger!</b><br /> <br /> So, how to do this the, seemingly to me, right way? I've looked at using the Passenger packets from Debian, but there seems to be no nginx module. I tried out the existing <tt>passenger</tt> package which offered to recompile nginx for me, at which point I most definitely lost interest. :-/ But Phusion's official repository has a proper <tt>libnginx-mod-http-passenger</tt> package! I'm not a big fan of third-party repos, but when I only need it on my master and not on the entire fleet, I'll survive.<br /> <br /> <a href="https://p.gaa.st/view/50719708">Here's most of my work, described in Puppet DSL (mind you, not directly reusable for others).</a> tl;dr on it: Adds the repo, pinned to be used only for packages with "passenger" somewhere in the name, installs the packages from it that I need, plus nginx (that one from Debian proper) with only the Puppet master service enabled using a <a href="https://p.gaa.st/view/91612911">snippet like this</a>.<br /> <br /> Only thing left to do is the file used as Passenger glue, and this is where the existing written instructions fall short, and where the Debian package wins: The default file is pretty bare whereas the Debian one contains a few tricks to rewrite Puppet 3 agent requests into something your Puppet 4 master will understand!<br /> <br /> So what I did: I've fetched the <a href="https://packages.debian.org/puppet-master-passenger">puppet-master-passenger</a> package, but instead of installing it (which pulls in Apache, etc.), I've taken out only the <tt>/usr/share/puppet/rack/puppet-master</tt> portion, and put that into <tt>/etc/puppet/rack</tt>. (This is where <tt>apt install -d</tt> and <tt>dpkg --extract</tt> come in handy.)<br /> <br /> One last thing here that is mentioned but not explained in other docs, yet very important: Ensure config.ru is owned by GID+UID <tt>puppet</tt>. As it turns out, that's the only thing instructing the Passenger framework what user to run the scripts as. I suppose implicit setuid behaviour like this is what the Ruby on Rails crowd considers elegant... :-)<br /> <br /> Anyway, that's all I needed to do to get the Puppet 3 agents to talk to the new master again. However, there are <a href="https://docs.puppet.com/puppetserver/latest/compatibility_with_puppet_agent.html">other known incompatibilities</a>, expect syntax errors or unwanted changes if you're not careful! Besides what's listed on that page, I ran into the issue of <tt>File</tt> objects that used to get created owned as <tt>root</tt> by default, now inherit the numeric UID+GID from the Puppet master. This was easily fixed with the following snippet in the site manifest (which is also where I learned one can get global defaults for object types this way!):<br /> <br /> <pre> # Sets the default for Puppet 3 agents. Should be safe to remove when # they're gone. # https://docs.puppet.com/puppet/3.8/deprecated_resource.html#default-copying-of-source-permissions File { source_permissions => ignore, } </pre>That's pretty much it, I think. Many thanks to Apollon Oikonomopoulos in <a href="http://lists.alioth.debian.org/pipermail/pkg-puppet-devel/2017-September/010880.html">this thread</a> for reminding me to use the right <tt>config.ru</tt> file. To those who dislike third party repositories, Georg Faerber's response on using uwsgi may also be interesting! Sat, 09 Sep 2017 19:19:00 +0000 https://wilmer.gaa.st/blog/archives/78-guid.html Dear /dev/null, https://wilmer.gaa.st/blog/archives/77-Dear-devnull,.html Life OSS https://wilmer.gaa.st/blog/archives/77-Dear-devnull,.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=77 1 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=77 nospam@example.com (Wilmer van der Gaast) That's what weblogs are these days, it seems. :-( And I've just lost way too much time getting this quality PHP software to run on my new webserver.<br /> <br /> All it is for me at this point is a place to post random small projects I've worked on. Something fun (ADS-B-related) will hopefully appear soon...<br /> Mon, 11 Jan 2016 00:44:00 +0000 https://wilmer.gaa.st/blog/archives/77-guid.html selphy.go - Linux client for Canon Selphy CP-900 photo printer https://wilmer.gaa.st/blog/archives/76-selphy.go-Linux-client-for-Canon-Selphy-CP-900-photo-printer.html Development https://wilmer.gaa.st/blog/archives/76-selphy.go-Linux-client-for-Canon-Selphy-CP-900-photo-printer.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=76 15 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=76 nospam@example.com (Wilmer van der Gaast) Last month I ran into a Canon Selphy CP-900 photo printer. It's a pretty neat device, prints pictures with pretty good quality. It even worked out of the box in Linux, but only via USB, and by it pretending to be an ordinary printer. This can have pretty ugly results.<br /> <br /> The CP-900, as the first one in the Selphy series, can also take print jobs over WiFi. The nice thing is, other than it being wireless, that the WiFi protocol is just a pretty simple JPEG file transfer mechanism. Just giving the printer a JPEG and have it figure out layout/crop/etc seems to give somewhat nicer results. But of course, only Windows/OSX/smartphone clients are available. Fortunately, after some staring at wireshark, I managed to create a new client for it.<br /> <br /> For extra fun, it's written in Go. My first time using the language, and I quite like it. It means the tool is very easy to build, and there are no special dependencies at all, as the Go base libraries already have everything I need.<br /> <br /> To try it: <pre>bzr branch http://wilmer.gaa.st/selphy/</pre>And yes, I'm still using bzr. Deal with it. :-)<br /> Sat, 21 Sep 2013 23:13:34 +0000 https://wilmer.gaa.st/blog/archives/76-guid.html T-4 days... https://wilmer.gaa.st/blog/archives/75-T-4-days....html https://wilmer.gaa.st/blog/archives/75-T-4-days....html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=75 0 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=75 nospam@example.com (Wilmer van der Gaast) Counting down and thinking "next week this time ..." for a while already. Predictably, I've only been able to do a fraction of all the things I wanted to do in the previous post. Me spraining my ankle and ending up on crutches a few weeks ago definitely didn't help with that. :><br /> <br /> The one-way flight will be on Thursday, giving us a whole Friday + a weekend before work starts. Hoping to get stuff like opening a UK bank account done quickly, so I can then figure out the cheapest way to transfer money there; I'll miss the perks of living in a Euro country...<br /> Sun, 22 Apr 2012 09:16:09 +0000 https://wilmer.gaa.st/blog/archives/75-guid.html Do you hear that, Doug? I'm coming to London! https://wilmer.gaa.st/blog/archives/74-Do-you-hear-that,-Doug-Im-coming-to-London!.html London https://wilmer.gaa.st/blog/archives/74-Do-you-hear-that,-Doug-Im-coming-to-London!.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=74 1 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=74 nospam@example.com (Wilmer van der Gaast) <a href="http://www.youtube.com/watch?v=wGbWJoZAsCs" class="bb-url">Yes, London. You know: fish, chips, cup 'o tea, bad food, worse weather, Mary fucking Poppins... LONDON. </a> Or as I said it on Twitter last week already, I'll be subtracting 309 from my international dialing code.<br /> <br /> My team is moving to London over the next months, and I'm moving with it. It's been five years already since I landed here in Ireland by now, longer than I ever expected when I arrived. Living in the UK instead of in Ireland will be a nice change. It's still off the European continent, but with directs flights to home for both of us we'll feel like we're much closer. And I expect there are more advantages that will offset living in what is also a heavily congested city.<br /> <br /> I'll still be doing <a href="http://code.google.com/speed/public-dns/" class="bb-url">the</a> <a href="http://tools.ietf.org/html/draft-vandergaast-edns-client-subnet-00" class="bb-url">same</a> <a href="https://www.google.com/search?q=google+global+cache" class="bb-url">work</a>, just in a different location for various practical reasons. The move will happen somewhere halfway the next quarter, so we're slowly starting preparations. Besides packing and cancelling utilities/services/etc, that also means experiencing Ireland as a tourist. After living here for five years, I've still not seen the Giant's Causeway, the Cliffs of Moher and some other stuff ... so should give that a shot now!<br /> <br /> I might be posting here a little bit more over the next while. :-)<br /> Sun, 04 Mar 2012 21:34:23 +0000 https://wilmer.gaa.st/blog/archives/74-guid.html uhat, using your joystick's hat switch in Linux flight simulators https://wilmer.gaa.st/blog/archives/73-uhat,-using-your-joysticks-hat-switch-in-Linux-flight-simulators.html Development Linux Stuff that flies https://wilmer.gaa.st/blog/archives/73-uhat,-using-your-joysticks-hat-switch-in-Linux-flight-simulators.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=73 4 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=73 nospam@example.com (Wilmer van der Gaast) So I have this fun hobby for a while already, flying.. I have around 50 hours logged by now in the US + Ireland, which means I can more or less land safely now, on my own. In fact I had my first solo in October last year which was an absolutely amazing experience. But sometimes weather just doesn't work with me here in Ireland (either too windy or too cloudy) and instead I go "flying" with X-Plane on my machine at home.<br /> <br /> Now X-Plane is a pretty neat simulator, and as long as you use it with a real yoke/stick and not keyboard/mouse, it seems like a useful way to practice. But there's one way in which a flight simulation projected on a single screen, no matter its size, just doesn't beat sitting in a cockpit: the inability to look around in any direction by, you know, just turning your head. Instead, joysticks often have this <a href="http://marsxplr.com/_upl/aviatorlogitechjse-orig.jpg" class="bb-url">hat switch</a> on the top to look around. Unfortunately in Linux, the joystick driver gets told that the hat switch is a mini-joystick that the user can move up/down, left/right. Instead of just representing it as four separate buttons (which is what they really are anyway, hardware-wise). X-Plane and apparently other flight simulators can't use this, they need buttons.<br /> <br /> This week I wrote <a href="http://wilmer.gaa.st/main.php/uhat.html" class="bb-url">uhat</a> to solve this problem. It'll listen to joystick events and if you move the hat switch axes, it will generate button events on a separate virtual joystick device. There's a similar tool called jhat, which generates keyboard events instead, but I never really liked the idea of my joystick pretending to be a keyboard and hoped there were a better way to do this. A week ago I found my answer in uinput. It's poorly documented, but fortunately very simple to figure out. It looks like uinput is just a fairly 1:1 translation of the input subsystem kernel interface into a character device.<br /> <br /> It works like a charm for me, with the udev rule I don't even have to think about it, udev will just start it for me when I plug in my joystick. Hugely enjoying X-Plane 10 again. :-D<br /> Sat, 03 Mar 2012 13:23:00 +0000 https://wilmer.gaa.st/blog/archives/73-guid.html Debian, dmcrypt and SSD TRIMming https://wilmer.gaa.st/blog/archives/72-Debian,-dmcrypt-and-SSD-TRIMming.html Linux https://wilmer.gaa.st/blog/archives/72-Debian,-dmcrypt-and-SSD-TRIMming.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=72 4 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=72 nospam@example.com (Wilmer van der Gaast) Spent an hour or so this morning wondering how to get my Debian initramfs to activate my LUKS-encrypted partition with --allow-discards. I know it's less secure, but as long as <a href="http://xkcd.com/538/" class="bb-url">wrenches are still cheap</a> I'm fine with sub-standard security if it means my hardware will perform better for longer. :-)<br /> <br /> The trick is to add a flag "discard" to your crypttab, like this:<br /> <br /> <div class="bb-code-title">CODE:</div><div class="bb-code">wilmer@peer&#58;~$&#160;cat&#160;/etc/crypttab<br /> sda2_crypt&#160;/dev/sda2&#160;none&#160;luks,discard</div><br /> <br /> And then of course rebuild your initramfs (update-initramfs -u) and reboot, etc.<br /> <br /> You do need cryptsetup 1.4 or higher for this to work. I had to manually install that package (only <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=648868" class="bb-url">twenty or so days old</a>) from sid on my testing laptop.<br /> Mon, 27 Feb 2012 22:44:36 +0000 https://wilmer.gaa.st/blog/archives/72-guid.html Bank statement scraper for Bank of Ireland https://wilmer.gaa.st/blog/archives/71-Bank-statement-scraper-for-Bank-of-Ireland.html Development https://wilmer.gaa.st/blog/archives/71-Bank-statement-scraper-for-Bank-of-Ireland.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=71 0 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=71 nospam@example.com (Wilmer van der Gaast) Like many people, I was also losing track of my finance. Having bank accounts in use in both NL and IE probably didn't help. :-) As any proper FOSS geek, I learned to like the monster called <a href="http://www.gnucash.org/" class="bb-url">GnuCash</a>. (Psst! Guys! It's pretty amazing that a product more than ten years old still doesn't let you do operations (like delete) on multiple entries at once, dont you think?)<br /> <br /> And there's this thing about Irish Banks. They have <a href="http://en.wikipedia.org/wiki/2008%E2%80%932011_Irish_banking_crisis" class="bb-url">bigger issues</a> to worry about than how well their Internet banking service works. What keeps you away from looking at my bank account? You (hopefully) not knowing my six-digit user ID, date of birth (top secret information! Have I mentioned that my birthday is next Saturday? ;-P) and another six-digit number, this time my PIN number. No one-time passwords, no challenge-response system, nothing else.<br /> <br /> My only hope is that this lets you transfer money only to accounts to which I've transferred money before. IOW all you can do is give my landlady a little present. Pfew!<br /> <br /> Also, going back to the original topic, there's no way to export info from their web interface. So I wrote one myself. One advantage of a pretty simple website is that I could easily write <a href="http://wilmer.gaa.st/main.php/boiscrape.html" class="bb-url">a scraper</a> for it. Run it with the right arguments, and it'll spit out a CSV bank statement, ready to be fed to your favourite accounting software.<br /> <br /> What else have I been doing? Been working on <a href="http://wilmer.gaa.st/main.php/giggity.html" class="bb-url">Giggity</a>. Android development's fun. I spent the weekend scraping the <a href="http://www.dancevalley.com/2011/#nl/news/volledige-timetable-bekend.html" class="bb-url">Dance Valley</a> timetable page, Google, Last.FM, Wikipedia and more to automatically generate a Giggity schedule file for it. Love it! :-)<br /> Mon, 01 Aug 2011 15:14:35 +0000 https://wilmer.gaa.st/blog/archives/71-guid.html On Pandaboard SD card performance https://wilmer.gaa.st/blog/archives/70-On-Pandaboard-SD-card-performance.html OSS https://wilmer.gaa.st/blog/archives/70-On-Pandaboard-SD-card-performance.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=70 3 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=70 nospam@example.com (Wilmer van der Gaast) I have the Pandaboard running as my home server for a while now. Until last weekend, I was using a Microdrive as its root filesystem. Sadly, the drive seems to be broken. :-( That means I finally had a chance to try bootstrapping a server very quickly using Puppet. This worked fairly well, which means the time investment is paying off already.<br /> <br /> Since all the storage I had at home was the 32GB SD card I bought for this thing anyway, I decided to give it another chance. At some point I was reminded already that <span style="font-weight:bold">alignment <span style="font-style:italic">really</span> matters</span> with these things. Some Bonnie++ runs do seem to confirm this. I removed the second partition on the SD, and recreated it on a 4MB barrier. (The trick to do this is to use the "u" command in fdisk to switch units to sector instead of cylinders, and make sure the start sector is a multiple of 8192.)<br /> <br /> To be honest, I did run most of these benchmarks with the SD card reader/writer in my desktop machine. Only the last test was done on my Pandaboard, but as you can see the results are very similar.<br /> <br /> <STYLE type="text/css">TABLE.small TD {font-size: 66%; padding: 0;} TABLE.small { border-width: 0 0 1px 1px; border-style: solid; } TABLE.small TD { border-width: 1px 1px 0 0; border-style: solid; } TD.header {text-align: center; backgroundcolor: "#CCFFFF" }TD.rowheader {text-align: center; backgroundcolor: "#CCCFFF" }TD.size {text-align: center; backgroundcolor: "#CCCFFF" }TD.ksec {text-align: center; fontstyle: italic }</STYLE><TABLE CELLSPACING=0 CLASS="small" ALIGN=center BORDER=3 CELLPADDING=2 CELLSPACING=1><TR><TD COLSPAN=2 class="header"><FONT SIZE=+1><B>Version 1.96</B></FONT></TD><TD COLSPAN=6 class="header"><FONT SIZE=+2><B>Sequential Output</B></FONT></TD><TD COLSPAN=4 class="header"><FONT SIZE=+2><B>Sequential Input</B></FONT></TD><TD COLSPAN=2 ROWSPAN=2 class="header"><FONT SIZE=+2><B>Random<BR />Seeks</B></FONT></TD><TD COLSPAN=1 class="header"></TD><TD COLSPAN=6 class="header"><FONT SIZE=+2><B>Sequential Create</B></FONT></TD><TD COLSPAN=6 class="header"><FONT SIZE=+2><B>Random Create</B></FONT></TD></TR><TR><TD></TD><TD>Size</TD><TD COLSPAN=2>Per Char</TD><TD COLSPAN=2>Block</TD><TD COLSPAN=2>Rewrite</TD><TD COLSPAN=2>Per Char</TD><TD COLSPAN=2>Block</TD><TD>Num Files</TD><TD COLSPAN=2>Create</TD><TD COLSPAN=2>Read</TD><TD COLSPAN=2>Delete</TD><TD COLSPAN=2>Create</TD><TD COLSPAN=2>Read</TD><TD COLSPAN=2>Delete</TD></TR><TR><TD COLSPAN=2></TD><TD class="ksec"><FONT SIZE=-2>K/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>K/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>K/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>K/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>K/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD COLSPAN=1></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD><TD class="ksec"><FONT SIZE=-2>/sec</FONT></TD><TD class="ksec"><FONT SIZE=-2>% CPU</FONT></TD></TR><TR><TD rowspan=2 bgcolor="#FFFFFF" class="rowheader"><FONT SIZE=+1>ext3-noalign</TD><TD class="size" bgcolor="#FFFFFF">4G</TD><TD bgcolor="#EB1400">189</TD><TD bgcolor="#2FD000">41</TD><TD bgcolor="#FF0000">3514</TD><TD bgcolor="#7B8400">1</TD><TD bgcolor="#C43B00">4878</TD><TD bgcolor="#08F700">1</TD><TD bgcolor="#0AF500">2646</TD><TD bgcolor="#03FC00">98</TD><TD bgcolor="#34CB00">21147</TD><TD bgcolor="#04FB00">2</TD><TD bgcolor="#F90600">20.9</TD><TD>0</TD><TD class="size" bgcolor="#FFFFFF">16</TD><TD bgcolor="#FF0000">463</TD><TD>0</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#E71800">2832</TD><TD bgcolor="#00FF00">3</TD><TD bgcolor="#CA3500">3214</TD><TD bgcolor="#00FF00">4</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#D12E00">3221</TD><TD bgcolor="#04FB00">4</TD></TR><TR><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#FF0000" COLSPAN=2>6643ms</TD><TD bgcolor="#E51A00" COLSPAN=2>28833ms</TD><TD bgcolor="#FF0000" COLSPAN=2>19841ms</TD><TD bgcolor="#03FC00" COLSPAN=2>6668us</TD><TD bgcolor="#FF0000" COLSPAN=2>483ms</TD><TD bgcolor="#EA1500" COLSPAN=2>47888ms</TD><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#01FE00" COLSPAN=2>1335us</TD><TD bgcolor="#00FF00" COLSPAN=2>655us</TD><TD bgcolor="#08F700" COLSPAN=2>933us</TD><TD bgcolor="#01FE00" COLSPAN=2>531us</TD><TD bgcolor="#0DF200" COLSPAN=2>85us</TD><TD bgcolor="#1CE300" COLSPAN=2>60814us</TD></TR><TR><TD rowspan=2 bgcolor="#FFFFFF" class="rowheader"><FONT SIZE=+1>ext3</TD><TD class="size" bgcolor="#FFFFFF">4G</TD><TD bgcolor="#AF5000">437</TD><TD bgcolor="#2DD200">93</TD><TD bgcolor="#FC0300">3727</TD><TD bgcolor="#6F9000">1</TD><TD bgcolor="#A35C00">5631</TD><TD bgcolor="#00FF00">1</TD><TD bgcolor="#00FF00">2725</TD><TD bgcolor="#01FE00">99</TD><TD bgcolor="#30CF00">21484</TD><TD bgcolor="#04FB00">2</TD><TD bgcolor="#F90600">21.5</TD><TD>0</TD><TD class="size" bgcolor="#FFFFFF">16</TD><TD bgcolor="#D82700">938</TD><TD bgcolor="#00FF00">1</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#57A800">4303</TD><TD bgcolor="#06F900">5</TD><TD bgcolor="#B74800">3743</TD><TD bgcolor="#07F800">6</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#B44B00">4304</TD><TD bgcolor="#00FF00">5</TD></TR><TR><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#10EF00" COLSPAN=2>412ms</TD><TD bgcolor="#FF0000" COLSPAN=2>32024ms</TD><TD bgcolor="#40BF00" COLSPAN=2>5622ms</TD><TD bgcolor="#00FF00" COLSPAN=2>6422us</TD><TD bgcolor="#F80700" COLSPAN=2>469ms</TD><TD bgcolor="#FF0000" COLSPAN=2>52114ms</TD><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#01FE00" COLSPAN=2>1574us</TD><TD bgcolor="#04FB00" COLSPAN=2>760us</TD><TD bgcolor="#00FF00" COLSPAN=2>713us</TD><TD bgcolor="#01FE00" COLSPAN=2>1644us</TD><TD bgcolor="#00FF00" COLSPAN=2>13us</TD><TD bgcolor="#01FE00" COLSPAN=2>585us</TD></TR><TR><TD rowspan=2 bgcolor="#FFFFFF" class="rowheader"><FONT SIZE=+1>logfs</TD><TD class="size" bgcolor="#FFFFFF">4G</TD><TD bgcolor="#00FF00">1149</TD><TD bgcolor="#00FF00">75</TD><TD bgcolor="#51AE00">13670</TD><TD bgcolor="#B34C00">5</TD><TD bgcolor="#FF0000">3530</TD><TD bgcolor="#FF0000">4</TD><TD bgcolor="#CE3100">1067</TD><TD bgcolor="#5AA500">81</TD><TD bgcolor="#FF0000">5920</TD><TD bgcolor="#FF0000">7</TD><TD bgcolor="#FF0000">6.5</TD><TD bgcolor="#FF0000">3</TD><TD class="size" bgcolor="#FFFFFF">16</TD><TD bgcolor="#7F8000">2033</TD><TD bgcolor="#36C900">9</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#2CD300">4731</TD><TD bgcolor="#C63900">22</TD><TD bgcolor="#FF0000">1755</TD><TD bgcolor="#3CC300">8</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#FF0000">1516</TD><TD bgcolor="#FF0000">11</TD></TR><TR><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#00FF00" COLSPAN=2>10944us</TD><TD bgcolor="#08F700" COLSPAN=2>1810ms</TD><TD bgcolor="#32CD00" COLSPAN=2>4562ms</TD><TD bgcolor="#FF0000" COLSPAN=2>29505us</TD><TD bgcolor="#16E900" COLSPAN=2>54279us</TD><TD bgcolor="#06F900" COLSPAN=2>2401ms</TD><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#1EE100" COLSPAN=2>568ms</TD><TD bgcolor="#FF0000" COLSPAN=2>9300us</TD><TD bgcolor="#FF0000" COLSPAN=2>7881us</TD><TD bgcolor="#FF0000" COLSPAN=2>2783ms</TD><TD bgcolor="#FF0000" COLSPAN=2>1483us</TD><TD bgcolor="#FF0000" COLSPAN=2>570ms</TD></TR><TR><TD rowspan=2 bgcolor="#FFFFFF" class="rowheader"><FONT SIZE=+1>nilfs2</TD><TD class="size" bgcolor="#FFFFFF">4G</TD><TD bgcolor="#57A800">797</TD><TD bgcolor="#11EE00">95</TD><TD bgcolor="#C13E00">7118</TD><TD bgcolor="#788700">2</TD><TD bgcolor="#B64900">5186</TD><TD bgcolor="#05FA00">1</TD><TD bgcolor="#02FD00">2711</TD><TD bgcolor="#00FF00">98</TD><TD bgcolor="#0DF200">24112</TD><TD bgcolor="#01FE00">2</TD><TD bgcolor="#00FF00">617.3</TD><TD bgcolor="#09F600">29</TD><TD class="size" bgcolor="#FFFFFF">16</TD><TD bgcolor="#798600">2102</TD><TD bgcolor="#FF0000">36</TD><TD>+++++</TD><TD>+++</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#986700">4592</TD><TD bgcolor="#FF0000">71</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#00FF00">10886</TD><TD bgcolor="#A75800">56</TD></TR><TR><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#01FE00" COLSPAN=2>20454us</TD><TD bgcolor="#10EF00" COLSPAN=2>2767ms</TD><TD bgcolor="#1AE500" COLSPAN=2>2788ms</TD><TD bgcolor="#D82700" COLSPAN=2>25974us</TD><TD bgcolor="#05FA00" COLSPAN=2>21810us</TD><TD bgcolor="#0BF400" COLSPAN=2>3432ms</TD><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#FF0000" COLSPAN=2>4928ms</TD><TD bgcolor="#13EC00" COLSPAN=2>1288us</TD><TD bgcolor="#0BF400" COLSPAN=2>1020us</TD><TD bgcolor="#01FE00" COLSPAN=2>1339us</TD><TD bgcolor="#3CC300" COLSPAN=2>358us</TD><TD bgcolor="#00FF00" COLSPAN=2>294us</TD></TR><TR><TD rowspan=2 bgcolor="#FFFFFF" class="rowheader"><FONT SIZE=+1>ext4</TD><TD class="size" bgcolor="#FFFFFF">4G</TD><TD bgcolor="#AD5200">443</TD><TD bgcolor="#2BD400">91</TD><TD bgcolor="#00FF00">18315</TD><TD bgcolor="#00FF00">2</TD><TD bgcolor="#00FF00">9299</TD><TD bgcolor="#0BF400">2</TD><TD bgcolor="#10EF00">2602</TD><TD bgcolor="#05FA00">99</TD><TD bgcolor="#00FF00">25041</TD><TD bgcolor="#00FF00">2</TD><TD bgcolor="#F90600">22.4</TD><TD>0</TD><TD class="size" bgcolor="#FFFFFF">16</TD><TD bgcolor="#00FF00">3567</TD><TD bgcolor="#13EC00">8</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#50AF00">4369</TD><TD bgcolor="#1EE100">7</TD><TD bgcolor="#B54A00">3792</TD><TD bgcolor="#0BF400">7</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#B14E00">4393</TD><TD bgcolor="#09F600">6</TD></TR><TR><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#01FE00" COLSPAN=2>32264us</TD><TD bgcolor="#00FF00" COLSPAN=2>920ms</TD><TD bgcolor="#00FF00" COLSPAN=2>926ms</TD><TD bgcolor="#48B700" COLSPAN=2>12851us</TD><TD bgcolor="#00FF00" COLSPAN=2>14137us</TD><TD bgcolor="#14EB00" COLSPAN=2>5042ms</TD><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#00FF00" COLSPAN=2>530us</TD><TD bgcolor="#18E700" COLSPAN=2>1451us</TD><TD bgcolor="#16E900" COLSPAN=2>1316us</TD><TD bgcolor="#00FF00" COLSPAN=2>393us</TD><TD bgcolor="#44BB00" COLSPAN=2>401us</TD><TD bgcolor="#01FE00" COLSPAN=2>771us</TD></TR><TR><TD rowspan=2 bgcolor="#FFFFFF" class="rowheader"><FONT SIZE=+1>ext4-panda</TD><TD class="size" bgcolor="#FFFFFF">1496M</TD><TD bgcolor="#FF0000">106</TD><TD bgcolor="#FF0000">96</TD><TD bgcolor="#1AE500">16838</TD><TD bgcolor="#FF0000">8</TD><TD bgcolor="#14EB00">8865</TD><TD bgcolor="#A45B00">7</TD><TD bgcolor="#FF0000">663</TD><TD bgcolor="#FF0000">99</TD><TD bgcolor="#26D900">22243</TD><TD bgcolor="#6B9400">12</TD><TD bgcolor="#F60900">30.1</TD><TD bgcolor="#00FF00">1</TD><TD class="size" bgcolor="#FFFFFF">16</TD><TD bgcolor="#08F700">3479</TD><TD bgcolor="#669900">26</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#718E00">4035</TD><TD bgcolor="#FF0000">23</TD><TD bgcolor="#00FF00">8760</TD><TD bgcolor="#6F9000">65</TD><TD>+++++</TD><TD>+++</TD><TD bgcolor="#0EF100">10388</TD><TD bgcolor="#CE3100">63</TD></TR><TR><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#03FC00" COLSPAN=2>78219us</TD><TD bgcolor="#19E600" COLSPAN=2>3925ms</TD><TD bgcolor="#01FE00" COLSPAN=2>961ms</TD><TD bgcolor="#51AE00" COLSPAN=2>13732us</TD><TD bgcolor="#31CE00" COLSPAN=2>104ms</TD><TD bgcolor="#00FF00" COLSPAN=2>1251ms</TD><TD class="size" bgcolor="#FFFFFF" COLSPAN=1>Latency</TD><TD bgcolor="#01FE00" COLSPAN=2>1190us</TD><TD bgcolor="#22DD00" COLSPAN=2>1801us</TD><TD bgcolor="#2AD500" COLSPAN=2>1892us</TD><TD bgcolor="#01FE00" COLSPAN=2>762us</TD><TD bgcolor="#09F600" COLSPAN=2>61us</TD><TD bgcolor="#01FE00" COLSPAN=2>671us</TD></TR></TABLE><br /> Click <a href="http://roy.gaast.net/~wilmer/bonniepp.html" class="bb-url">here</a> for a table not f*cked up by my blog software.<br /> <br /> Although the throughput numbers for ext3 are pretty similar for non-aligned and aligned access, look at the latency numbers. Unfortunately I haven't got a clue how Bonnie++ calculates these and can't find very good documentation on it. Throughput may be average and latency worst-case? Either way, as you can see a misaligned partition can cause some slowdowns.<br /> <br /> What surprised me more is that a switch to ext4fs sped up things a lot more, up to the point that the performance is perfectly reasonable! I'm running with this SD as my root filesystem now and everything just works. (While before a simple apt-get install run could take several minutes.)<br /> <br /> While I was at it, I also tried out logfs and nilfs2, which are officially optimised for flash media. However, AFAIK they're more meant for raw NAND storage, not for block devices with all the NAND logic abstracted away (like anything you buy in stores these days). Not worth it for these SDs.<br /> <br /> Obviously this test is far from scientific. Only in the case of ext4-panda have I run the test five times to then pick a decent result (there were some outliers in all areas). All other tests were done on a freshly formatted filesystem, which I'm sure also doesn't make the result that reliable.<br /> <br /> Just my 2 cents! But my Pandaboard's definitely happier now. Here's hoping that wear leveling works well..<br /> <br /> If you're interested, <a href="https://wiki.linaro.org/WorkingGroups/Kernel/Projects/FlashCardSurvey" class="bb-url">here</a> is a more thorough overview of SD card performance. The LWN article about flash storage it links to is interesting too. The Flash card I used here is a 32GB class 10 Transcend card.<br /> <br /> Mon, 13 Jun 2011 21:03:53 +0000 https://wilmer.gaa.st/blog/archives/70-guid.html Splitting PDFs with pyPdf https://wilmer.gaa.st/blog/archives/69-Splitting-PDFs-with-pyPdf.html https://wilmer.gaa.st/blog/archives/69-Splitting-PDFs-with-pyPdf.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=69 1 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=69 nospam@example.com (Wilmer van der Gaast) A simple task, yet I couldn't find a quick cmdline to do it with, apart from pdftk, 15MB of Java rubbish.<br /> <br /> Instead, here only 10 or so lines of Python. It was so fast I wasn't sure if it worked until I saw the results were there. Usage: split [prefix] [infiles...]. Multiple infiles possible. First argument is the filename prefix to use for all created files.<br /> <br /> <div class="bb-code-title">CODE:</div><div class="bb-code">import&#160;pyPdf<br /> import&#160;sys<br /> <br /> n&#160;=&#160;0<br /> for&#160;f&#160;in&#160;sys.argv&#91;2&#58;&#93;&#58;<br /> f&#160;=&#160;pyPdf.PdfFileReader&#40;open&#40;f&#41;&#41;<br /> for&#160;p&#160;in&#160;f.pages&#58;<br /> of&#160;=&#160;pyPdf.PdfFileWriter&#40;&#41;<br /> of.addPage&#40;p&#41;<br /> of.write&#40;open&#40;"%s-%03d.pdf"&#160;%&#160;&#40;sys.argv&#91;1&#93;,&#160;n&#41;,&#160;"w"&#41;&#41;<br /> n&#160;+=&#160;1</div><br /> <br /> Don't pay attention to Serendipity screwing up the code layout. We all know it's rubbish, I just can't be arsed to migrate to something better. :-/<br /> Fri, 03 Jun 2011 22:04:02 +0000 https://wilmer.gaa.st/blog/archives/69-guid.html dnsrev https://wilmer.gaa.st/blog/archives/68-dnsrev.html Development https://wilmer.gaa.st/blog/archives/68-dnsrev.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=68 1 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=68 nospam@example.com (Wilmer van der Gaast) As a bit of a cloud "sceptic" I still like to waste too much time maintaining my own network/IT infrastructure. :> I'm definitely trying to avoid the more tedious stuff though. I started using <a href="http://www.puppetlabs.com/" class="bb-url">Puppet</a> a while ago which definitely helps.<br /> <br /> Last week I was looking for a way to automatically populate DNS reverse lookup zones. The only thing I could find was <a href="http://www.mkrdns.org/" class="bb-url">mkrdns</a> which is unmaintained for almost ten years and doesn't seem to support IPv6. So I decided to write my own thing, <a href="http://gaa.st/dnsrev" class="bb-url">dnsrev</a>.<br /> <br /> It's pretty simple, written in Python with help from some modules. It can read any number of zonefiles and update any number of reverse zonefiles. There's no need for any kind of 1:1 mapping between them, so it can deal with multiple netblocks in one zonefile, etc. I hope it'll be useful to someone. Comments, suggestions and patches are welcome.<br /> Tue, 24 May 2011 00:09:14 +0000 https://wilmer.gaa.st/blog/archives/68-guid.html Shiny happy hardware https://wilmer.gaa.st/blog/archives/67-Shiny-happy-hardware.html OSS https://wilmer.gaa.st/blog/archives/67-Shiny-happy-hardware.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=67 2 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=67 nospam@example.com (Wilmer van der Gaast) For years I'm using <a href="http://winterm.gaast.net/" class="bb-url">Winterms</a> as simple home "servers". It was a fun project to work on and some people were even nice enough to send me some examples of more powerful (relatively, we're talking about ~300MHz here at most) hardware. Two of them are still working as nameservers/printservers and one of them even hosted the Winterm hacking website for a while.<br /> <br /> But they're getting old, slow, and pretty painful to upgrade. Time to move on I'm afraid. So before my last trip to the US, I ordered two shiny pieces of hardware: A <a href="http://www.pandaboard.org/" class="bb-url">Pandaboard</a> and an <a href="http://developer.nvidia.com/tegra/tegra-devkit-features" class="bb-url">Nvidia Tegra developer board</a>. Due to circumstances, I didn't really expect both (or even either) of them to arrive - Nvidia seemed to send the board only to people who have projects they find interesting/important (stuff like the <a href="http://en.wikipedia.org/wiki/Motorola_Xoom" class="bb-url">Motorola Xoom</a> probably), and the Pandaboards never seem to be in stock.<br /> <br /> Yet, here I am with both of them, wondering which one to actually use. :-)<br /> <br /> <a href="http://fotos.gaast.net/stuff/20110423_armboards.jpeg" class="bb-url"><img src="http://fotos.gaast.net/stuff/20110423s_armboards.jpeg" alt="http://fotos.gaast.net/stuff/20110423s_armboards.jpeg" class="bb-image" /></a><br /> Left: Pandaboard, right: Tegra2 250 Harmony board<br /> <br /> I guess I'll just write down my findings here so far. I'll probably end up using both, one as a server and the other one to run stuff like <a href="http://www.xbmc.org/" class="bb-url">xbmc</a> on my TV.<br /> <br /> Both boards seem quite similar, spec-wise. Two 1GHz ARM cores, 1G of RAM, USB, sound, networking (including WiFi and Bluetooth), HDMI output, and an SD card slot. The Pandaboard has an internal antenna, no clue about the range.<br /> <br /> Although both boards' USB ports apparently aren't really meant for powering 2.5" USB HDDs, it seems to work quite well anyway. Which is good, because SD cards as root filesystems seems like a bad idea. Did you know that (according to <a href="http://www.coker.com.au/bonnie++/" class="bb-url">bonnie++</a>) a desktop hard disk from 2007 outperforms SD cards (at least in the Pandaboard and Tegra) not just on sequential reads, but also on seeks? So yeah, I may be using USB HDDs instead, which sadly means more power usage. :-( Especially in the Pandaboard SD performance is too bad to be usable.<br /> <br /> One big advantage of the Pandaboard seems to be the community. A pretty busy (and generally helpful) IRC channel, lots of info online on Wikis. The Pandaboard iss "just another OMAP architecture" so lots of stuff that worked for BeagleBoard should work on the Panda with some customizations. Canonical/Ubuntu also <a href="https://wiki.ubuntu.com/ARM/OMAP" class="bb-url">support</a> the thing officially.<br /> <br /> Here comes the biggest contrast with the Tegra. Nvidia seems to be <a href="http://developer.nvidia.com/tegra/forum/linux-support-becoming-unsupported" class="bb-url">too busy with Android</a>, the result is that there's little support for doing other stuff with the board. The only thing you get for now is L4T (Linux 4 Tegra), which is an Ubuntu Jaunty (yes, 9.04, that's two years ago by now..) image you can run on it. There are efforts on getting Lucid to run, don't know where those are ATM. But one complication there is some binary-only drivers/helpers (like nvrm_daemon, which I guess manages the memory shared between OS and video/etc), which means troubles getting X to work after an upgrade. Ouch.<br /> <br /> The Panda also certainly wins in the bootloader department, as it just loads uboot stuff from a FAT partition on the SD card (tricky part here is that if you do <strong>anything</strong> wrong with the partitioning and formatting of this SD card, the boot process will just fail silently). For flashing the Tegra you need a proprietary fastboot flasher binary. Possibly, once booted, I can just write my kernels to NAND myself from inside the OS, but I haven't yet tried this.<br /> <br /> So yes, with this all in mind, it's a delight to run a normal (and not outdated) Debian/Ubuntu install on a Pandaboard. Video is also supposed to work flawlessly almost out of the box on Ubuntu. However, I seem to be unlucky/doing it wrong since the framerate is not impressive, and playback seems buggy. (While the little video playback I've done on the Tegra was pretty good, super smooth, and with only 10% of CPU usage!)<br /> <br /> For my original goal, running a simple home server, I feel that both boards are suitable - I'd just run Debian inside a chroot on the Tegra so the helper daemons (and maybe some video stuff) can run outside it. But before I get this video stuff to work, I have some work to do. And hopefully, if I wait for long enough, some other patient souls out there will also fix some of these problems...<br /> Sat, 23 Apr 2011 13:05:00 +0000 https://wilmer.gaa.st/blog/archives/67-guid.html Xen horror, just upgraded my box to Squeeze https://wilmer.gaa.st/blog/archives/66-Xen-horror,-just-upgraded-my-box-to-Squeeze.html https://wilmer.gaa.st/blog/archives/66-Xen-horror,-just-upgraded-my-box-to-Squeeze.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=66 3 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=66 nospam@example.com (Wilmer van der Gaast) Of course I could just <strong>not</strong> upgrade, but I'd have to do it sooner or later anyway..<br /> <br /> It looks like Xen, now that it's owned by Citrix, also suffers from the <a href="http://bugs.bitlbee.org/bitlbee/browser/devel/protocols/msn/soap.c#L8" class="bb-url">XML manager syndrome</a>. At least the extremely annoying <a href="http://www.google.com/search?q=%22clocksource%2F0%3A+Time+went+backwards%22" class="bb-url">clock bug</a> was fixed, which means I get >4ms precision in timing again.<br /> <br /> Just dumping this here in my blog since more people seem to have this problem and aren't getting very helpful answers so far. Or maybe I just didn't try the right Google queries..<br /> <br /> This:<br /> <pre>hypnotoad:/tmp# xm suspend bijtje<br />Error: Domain is not managed by Xend lifecycle support.<br />Usage: xm suspend &lt;DomainName> Suspend a Xend managed domain</pre>means you're not using the shiny new (whatever the purpose is) lifecycle tool that keeps track of domain configs and other stuff in /var/lib/xend/domains/$UUID/. You can use "xm new" to set this up, except this is broken on Debian:<br /> <pre>hypnotoad:/tmp# xm new<br />Unexpected error: &lt;type 'exceptions.ImportError'><br />...<br />ImportError: No module named xmlproc</pre>This, my dear reader, means that Debian's shipping Xen utils that depend on a Python XML module that was actually removed from Debian since it's not maintained by upstream anymore.<br /> <br /> So I was just trying to figure out how to make this all go, and then I realised:<br /> <pre>hypnotoad:/tmp# xm save bijtje bijtje.sav<br />hypnotoad:/tmp# xm restore bijtje.sav</pre>Oh look! I can still do it. I just have to tell Xen where to save the statefile.<br /> <br /> So in short:<br /> <ul><li>Xen is also suffering from XML-itus<br /> <li>Debian drops packages that other packages still depend on<br /> <li>Use "xml save", not "xml suspend"</ul><br /> Sun, 13 Feb 2011 23:49:48 +0000 https://wilmer.gaa.st/blog/archives/66-guid.html Burn all spammers! https://wilmer.gaa.st/blog/archives/65-Burn-all-spammers!.html https://wilmer.gaa.st/blog/archives/65-Burn-all-spammers!.html#comments https://wilmer.gaa.st/blog/wfwcomment.php?cid=65 0 https://wilmer.gaa.st/blog/rss.php?version=2.0&type=comments&cid=65 nospam@example.com (Wilmer van der Gaast) I have a habit of always having a tail -f /var/log/mail.log running on my mailserver somewhere. It's noisy, but has been useful in the past. Over the last weeks/months, I noticed open relay probes are getting incredibly popular (again), but also extremely aggressive. They're frequent, done by hundreds of botnet drones all the time.<br /> <br /> Obviously my Postfix is configured properly, so this is mostly a waste of (fairly scarce, on a DSL box several km away from the exchange) bandwidth and annoying noise in the logs. But getting rid of it is harder than I hoped. :-(<br /> <br /> This is what I have now: iptables -I FORWARD -p tcp --sport 25 -s 192.168.0.0/16 -m string --algo kmp --string '554 5.7.1 <' -j REJECT --reject-with tcp-reset<br /> <br /> This works as-in it kills the connection as soon as my mailserver sends a "554 5.7.1 Relaying denied" response. The REJECT goes to the mailserver, but together with the tcp-reset this also kills the TCP connection on both sides fairly quickly. However, the little fuckers are also using pipelining, so I still get a screen full of logspam for pretty much every attempt. Although this is mostly cosmetic, I'd love to get rid of that crap..<br /> <br /> What I really wonder is, WTF are they even doing this? Are open relays really still that common? Don't they have their botnets already? I guess the open relays are nice multipliers and are also more willing to deal with stuff like graylisting...<br /> <br /> [edit]Looks like "554 5.7.1" is not just about "relaying denied", so possibly not such a great idea. Don't try this at home!<br /> Sun, 08 Aug 2010 14:31:06 +0000 https://wilmer.gaa.st/blog/archives/65-guid.html