Author Archives: Frank Groeneveld

Crawljax 1.8 released

Posted on by
Reply

I'm working on a cool project for my master thesis: Crawljax. Crawljax is a website crawler that supports JavaScript. This is done by opening a real browser such as Firefox and controlling it via WebDriver. The core of Crawljax does only that: crawling websites. However, there is a very flexible plugin system available that allows you to do all kinds of cool things such as creating a static mirror of an AJAX website or creating test suites for you AJAX webapplications.

Download Crawljax now and give it a try!

To get a better grasp of what is possible, have a look at the Google Tech Talk a colleague of mine did:

Update your HTC Hero radio

Posted on by
Reply

One week after I bought my Hero, I found out it is possible to install a newer radio-firmware on the HTC Hero. This has all kinds of positive improvements such as better reception and better battery life. I don't know where these radio updates come from (I guess from HTC themselves), but they aren't published on the HTC website. You can find them at the xda-developers forums (currently, there is a newer update that is not listed in that thread) and they can be installed by saving them to the SD card of your Hero as "update.zip". Next, turn off your phone and turn it on again by holding home+poweron. Enjoy the better reception and battery life!

HTC Hero without Sense

Posted on by
5

A few weeks ago I bought my first smartphone: The HTC Hero. It's been a great purchase since the beginning, but I was wondering how Android would "feel" without all the HTC customizations such as the SenseUI. So, I installed an AOSP (Android Open Source Project) ROM for the Hero. It was a build of the Android 2.0.1 sources combined with the Google closed-source applications such as Gmail, Talk and Calendar. This was a really great experience, because all the features I use worked great. There was just one problem that is quite important for me: battery life. Somehow this ROM seems to drain the battery at night. When I first brought up this issue, nobody had noticed it, but quickly other people started to monitor their Hero's and it appeared they were experiencing the same problem. I've switched back to the official ROM until this is solved, but you should really give this a try. The developers of the ROM did a great job!

Howto: Live migrate to software RAID 1

Posted on by
Reply

Feel insecure about your data? Don't trust your harddrive anymore? Use this howto to migrate your running Ubuntu Linux system to software RAID 1.

Before starting off, I assume you have your running system on /dev/sda and your new harddrive is called /dev/sdb.

Boot up your system and install the mdadm package. We now have to create partitions on sdb that are the same as sda. Because I have identical disks, I just copy the partition table from sda to sdb like so:
sfdisk -d /dev/sda | sfdisk /dev/sdb
After that, I use partprobe to let the Linux kernel know I have changed the partition table.

Next step is to create a degraded RAID array on sdb which we can copy the files to and than add sda to the array. For every partition you have to run:
mdadm --create /dev/md0 --level 1 --raid-devices=2 missing /dev/sdb1

Where you replace sdb1 with the partition you want. Now create filesystems on these new raid devices:
mkfs.ext3 /dev/md0
Again, run this for all your mds.
After copying all the files to your new array, we have to modify a few files.

  • /boot/grub/menu.lst
  • /etc/fstab
  • /etc/initramfs-tools/conf/resume

All these files contain references to UUIDs that are no longer correct. I simply replaced them with /dev/md0 for example. You can try to use UUIDs, but I believe that the (striped) partitions have the same UUIDs as the raid devices (the mds). If you've changed everything, run:
mdadm --detail --scan >> /etc/mdadm/mdadm.conf
And don't forget to update your initrd:
initramfs -u

That's all! You can now enjoy the safety of RAID 1 without to much hassle. To check the status of your array, look in /proc/mdstat. Also make sure you setup a mail server or ssmtp, because the mdadm tools will try to send you an email if one of your RAID devices is degraded/corrupt.

Pinnacle DVB-T stick remote

Posted on by
Reply

Another post about that Pinnacle DVB-T stick? No, this one is about the remote! I wrote that the remote wasn't completely working yet. Apparently, there is a driver for this remote or the stick that converts the buttons from the remote to "keyboard events". Meaning I can type the number 0-9 with my remote for example. However, the "change channel" buttons appear to send a keycode that is above 255 and the X11 protocol only reserves one byte for keycodes. This means those keycodes can't be send to the X11 server and will disappear. There's a bug about this in the X.org bugzillay: http://bugs.freedesktop.org/show_bug.cgi?id=x11-keycode-limit and their solution: Change the X11 protocol (meaning: start development on the X12 protocol) or remap those keyevents somewhere between the kernel and X.org. I think the last solution is probably the simplest. I only have to figure out how to remap those keys with hal.

Pinnacle DVB-T stick on Ubuntu Jaunty

Posted on by
Reply

Yesterday I decided to buy the Pinnacle DVB-T Stick (also known as Pinnacle TV Stick 72e) to watch some free-to-air channels. In the shop I searched on Google for Linux support, to make sure I didn't have to return the product. The first few hits seemed positive (2 years ago), so I decided to take the plunge.

After arriving at home, I plugged the device in an empty USB port and saw this in my dmesg:


[11906.080060] usb 1-6: new high speed USB device using ehci_hcd and address 5
[11906.214249] usb 1-6: configuration #1 chosen from 1 choice
[11906.245886] dib0700: loaded with support for 8 different device-types
[11906.246212] dvb-usb: found a 'Pinnacle PCTV 72e' in cold state, will try to load a firmware
[11906.246222] usb 1-6: firmware: requesting dvb-usb-dib0700-1.20.fw
[11906.252816] dvb-usb: downloading firmware from file 'dvb-usb-dib0700-1.20.fw'
[11906.471526] dib0700: firmware started successfully.
[11906.972059] dvb-usb: found a 'Pinnacle PCTV 72e' in warm state.
[11906.972135] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[11906.972367] DVB: registering new adapter (Pinnacle PCTV 72e)
[11907.186126] DVB: registering adapter 0 frontend 0 (DiBcom 7000PC)...
[11907.369975] DiB0070: successfully identified
[11907.370122] input: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:12.2/usb1/1-6/input/input12
[11907.396132] dvb-usb: schedule remote query interval to 50 msecs.
[11907.396140] dvb-usb: Pinnacle PCTV 72e successfully initialized and connected.
[11907.396744] usbcore: registered new interface driver dvb_usb_dib0700

Great! All drivers and firmware load automatically in Ubuntu 9.04 Jaunty.

Next step: actually getting video on my screen. After Googling I found out I first had to scan for channels using dvbscan. However, it gave me the non-informal message "Unable to query frontend status". Apparently there is also another scanning program called scan. I ran it like this:
scan /usr/share/dvb/dvb-t/nl-All > ~/.mplayer/channels.conf
Now I could start mplayer dvb:// and enjoy watching DVB-T digital television! Note you can switch channels with h (next) and k (previous). Last thing I need to figure out is how to use the remote for this (I can change the volume already).

Fixing FreeNX on Fedora 11

Posted on by
Reply

Today I had some problems getting FreeNX to work on Fedora 11. FreeNX is a VNC-like system that has much better performance than VNC itself. It is therefore very usefull if you would like to access your computer via your Cable or ADSL connection.

After installing FreeNX in Fedora, whenever I connected, I got some strange error messages. The error message also told me to run some command via ssh to find out more. Apparently, FreeNX wanted to start on display port :0, which was already in use by X. To fix this, I copied /etc/nxserver/node.conf.sample to /etc/nxserver/node.conf and uncomment the line that says: DISPLAY_BASE=1000 and change it to DISPLAY_BASE=1001. This will start the NX server on port 1001 (display port :1).
However, this fix would still not allow me to connect. After searching around on the internet, I found out the package was missing a dependency on xorg-x11-fonts-misc. Installing it allowed me to open a session and enjoy the responsiveness of FreeNX!

In short: Make sure your DISPLAY_BASE port is not set to 1000 if you also have another X server running and don't forget to install xorg-x11-fonts-misc.

Using GNU screen as virtmanager for KVM

Posted on by
Reply

After my last post about KVM, somebody emailed me, asking how I use KVM's serial console in combination with screen. It's a rather simple, but really usefull solution if you think tools like virtmanager are too much for you and you don't like to use VNC. So this post will try to explain it to you.
For starters, you have to add the -nographic option to you KVM parameters. Next, make sure your virtual machine outputs it's console to serial device 0. For example, start the Linux kernel with console=ttyS0 and make sure there is a getty process running on that same device. Now, whenever you start KVM in a GNU screen (with that -nographic parameter) you will see the dmesg scroll by and will end up with a login prompt. You can just keep this KVM running while detaching the GNU screen session (ctrl+a d) and you can re-attach by starting screen -x.
I use this on my server to start my two virtual machines. I made a screenrc like this:
# display a nice status bar on the bottom of the screen
hardstatus alwayslastline "%-Lw%50>%n%f* %t%{-}%+Lw%< %=[%c]"
vbell off
deflogin off
# use virtio for disks also!
screen -t production kvm -m 2048m -nographic -drive file=production.raw,if=virtio,boot=on -net nic,model=virtio -net tap,ifname=tap0,script=no,downscript=no -smp 2
screen -t development kvm -m 1024m -nographic -drive file=development.raw,if=virtio,boot=on -net nic,model=virtio -net tap,ifname=tap1,script=no,downscript=no

Whenever the server boots it runs the following command in my rc.local:

# start KVM
cd /srv/
su frank - -c 'screen -c screenrc -dm'

So the server starts the GNU screen and the GNU screen start my virtual machines. If I can't login via ssh in one of my virtual machines, I just connect to the host and attach to the screen to debug the problem.

Speeding up your website

Posted on by
Reply

A few days ago I stumbled upon the new Google "Let's make the web faster"-page. I found some useful tips on there. Some of them are public knowledge, for example the fact that using echo in php is faster than print, single quotes are faster than double quotes and echo supports endless arguments, which is faster than string concatenating (so you write echo 'this', $is, 'faster than', $concatenating, 'the string' instead of echo 'this' . $is . 'slower').
However, there's a nice article about ommitting html tags, which had some tips I did not know. For example, if you're using HTML, instead of XHTML, you're allowed to ommit more tags than most people know. For example, you don't need to close a paragraph, you can just start a new one. For the complete list, have a look at the articles section of the site.