When is a scrum not a scrum?

As the craze to claim that a project is agile gathers pace, I see more cases where the daily meeting is referred to as a scrum. Gimme a break. See this article at the Agile Forest, for what is wrong with the trend in status meetings.
Quoting from the article:

The key purpose of a Stand-up is the opportunity to collaborate, share and support each other in the delivery of valuable outcomes.

I reckon that over time we will all see more and more examples of where the language of agile is adopted without the actual corresponding practice. There will be talk of sprints, iterations, retrospectives, scrums, kanbans, etc etc whilst the waterfall approach continues to be used on the project as the actual main method. Of course this means the project will not be able to embrace change since it will not be able to move quickly and easily in any direction. But then no-one will see anything wrong with that because waterfall is always the way it has been done. And a slow, plodding speed of delivery with document+design,code,test,signoff in strict order with no feedback will continue to be the way that software development is expected to be done.

At the next "scrum" I am tempted to ask who is the scrum master. But I have a feeling that the irony will be lost: I will just be told it is the person that asks each person in turn, "what did you do yesterday, what will you do today?" whilst the rest of team waits for the meeting to finish.

Migrating my web hosting

I have been with FastHosts for years but this was because they were (relatively) cheap and UK-based. However, I have just moved to BlueHost, which (IMHO) is far far better.

Switching over

FastHosts has been ok but they do not (at the time of writing this) provide any Postgres facility. Databases are MySQL only. I actually want/need both. I use MySQL for Coppermine, but would like to use Postgres for DSpace. There are not many UK-based providers that have the Postgres option. The option is much more common with US hosting companies.

There is another reason for preferring my hosting to be done outside Europe. A new stupid law has been passed that forces web sites to declare if they use cookies. Yeah, right. Problem solved. My website is now outside Europe.

I did a bit of research and found that BlueHost has everything and is reasonably priced. I moved my website over and had a few slight technical issues at first. I used their online 24/7 technical help and found it to be excellent and free of charge. It is also available 24/7. Most UK support is only in office hours Monday to Friday and you have to pay for it via expensive phone calls with lengthly waiting times and obnoxious musak.

BlueHost do not allow a .co.uk domain to be transferred to them. However, they can still host such domains. I found this a bit confusing at first. Domain registration is handled by certain official bodies across the world. In the UK a common organisation that does this is called Nominet. Web hosting providers that offer to handle your domain registration do so by dealing with Nominet. BlueHost do not deal with Nominet since Nominet is for the UK. However, that doesn't matter. What matters is how requests that are aimed at your domain get directed to the appropriate IP address. This is done using nameservers. When Fasthosts first set up my domain they configured my domain to use their own nameservers. But their webpages have configuration that allows this to be changed for when you wish to change who is hosting your domain. I changed them and in less than 12 hours my web site had changed over. Marvellous!

Migrating Coppermine

I hit a few snags when migrating my photos over to BlueHost. I use Coppermine to manage my photos, so I got the latest version, installed it on my own machine, set up a MySQL database and Apache with PHP, and started up the Coppermine landing page to verify that it would work properly.

When you land on a Coppermine home page for the first time, the PHP starts to configure your environment. It did this for me on my own machine (eventually) so I reasoned that the time was right for me to FTP all the files to my BlueHost area and it should work in just the same way. It didn't. It gave a weird PHP scripting error. Googling for the error message gave hits that made me think the problem was to do with the use of SSI (Server Side Includes). However, with some help from BlueHost support it was determined that the problem was to do with file permissions. Once the folders were set to 0755 and the files were set to 0644 all was well.

Getting a TV card working with Debian

I made the mistake of buying a card without first checking that it would work with linux. How foolish. I was forced to replace it. I now have a DVICO FusionHDTV DVB-T Dual Digital 4. I updated my system to Debian Wheezy (testing), 64 bit.

dmesg reveals the error:

[ 1501.907761] xc2028 1-0061: Error: firmware xc3028-v27.fw not found.

this problem is described at http://www.linuxtv.org/wiki/index.php/Xceive_XC3028/XC2028#How_to_Obtain_the_Firmware
However, those instructions assume 32 bit. I am on 64 bit.

there seems to be a problem getting hold of the the firmware file.
I registered on ubuntuforums.org and searched for firmware file. There is a post that has the file as an attachment.

I unpacked it and copied it to /lib/firmware.

Then I rebooted. The dmesg error went away.

I used the scan command to find the channels:

scan -u /usr/share/dvb/dvb-t/uk-CrystalPalace >channel.conf

I then gave the command:

cp channels.conf /etc && cp channels.conf ~/.tzap

The command:

tzap -c /etc/channels.conf -r 'BBC ONE'

is interesting. It produced the following over and over again.

status 1e | signal c024 | snr fdfd | ber 00000000 | unc 00000000 | FE_HAS_LOCK
status 1e | signal c008 | snr fefe | ber 00000000 | unc 00000000 | FE_HAS_LOCK
status 1e | signal c014 | snr fefe | ber 00000000 | unc 00000000 | FE_HAS_LOCK

It is as if something else had the tuner resource.
However, as it turns out, this is what it is supposed to do. Starting another terminal session whilst tzap is still running enables me to give the command:

mplayer /dev/dvb/adapter0/dvr0 which now works!

So I now have TV working on my debian machine. All I need now is to get a PVR environment working. I will have a go with either Freevo or MythTV.

How to use lean/agile as a stick with which to beat the developer

This article, http://jimhighsmith.com/2011/11/02/velocity-is-killing-agility, by Jim Highsmith, says it all.

Ipods

My wife and I have an iPod each, I am ashamed to say. AFAIK it is the only way one can have an MP3 player with a capacity greater than 32GB. As soon as a large capacity MP3 player comes along that is not made by Apple, we will buy it immediately. We have been waiting for years and we are still waiting. I reckon Apple have some kind of monopoly on large capacity MP3 players. They must do, right? Why else does no-one else produce one?
We have the black iPod classic 160GB version. Brenda's is the older model, the one where it is possible to disable the European cap, and sadly mine is more recent where, for the moment, the cap cannot be disabled (keep working on it, guys!).
But it's not the European cap that causes the most irritation. It's the fact that proprietary software is needed to actually use it. We are Windoze-free in our house so this means the iPod comes with a problem: how does one put music on it? I have been using gtkpod (thanks, guys, what a great utility!) but I have found a problem with it. When one uses gtkpod to update two different iPods from the same desktop it seems that gtkpod can get confused regarding which model the iPod is. I managed to trash Brenda's iPod, i.e. get it into the state where music is on it but the iPod reckons it is not there. It reckoned the iPod was empty in fact. Eventually I managed to fix this by editing the file /media/IPOD/iPod_Control/Device/SysInfo, removing the line that refers to the model name. If you do this then gtkpod prompts for it and this time I put the correct value in.
When you read the various posts on the web about how to make the iPod recognise that music is there that has been placed by gtkpod, they all say you need to give the firewire id. What they don't tell you is that the model name also has to be accurate. If either is missing or wrong then you get this problem. During my attempts to diagnose and fix the problem I did a factory reset and wiped all the music from Brenda's iPod. Luckily I have all the music on an external USB drive so I will be able to restore it. But why does Apple create this pain for all it's users? Cowon does not bother creating proprietary software for its MP3 players. To load a Cowon player you just drag and drop. Simple, eh? What a pity that the largest capacity MP3 player by Cowon is a mere 32GB (the S9).
I am patiently waiting for RockBox to support the iPod classic. I use RockBox on my ancient iRiver H30 and it is fabulous. What a pity I can't use it on the iPod yet. What a pity that I am not technical enough to help with the porting. The code really is quite hairy to a novice such as me. I know, I've looked.
I can't wait for Cowon to create a large capacity player. They have a tablet that does the job (the X7) but the tablet is just that little bit too large for Brenda to take to the gym. But the spec is great. It even plays FLAC and Ogg Vorbis, which of course, Apple does not do. Come on Cowon! Just go that little bit further and I'm sure your sales will skyrocket!

Ubuntu and I are no longer friends

I have decided to abandon Ubuntu. I don't know what I am going to use instead yet but with Ubuntu I have had the last straw. I have the same complaints about Canonical's direction as all the other developers but the straw that broke the camel's back was when Brenda said she didn't like it. Yes, that's right Canonical; even the computer-naive don't like it. And guess what; they don't like it for the same reasons that devs don't like it. I won't bother listing the reasons here. The web is already full of detailed complaints and it is not going to change the minds of those at Canonical. What worries me is how much of this is down to Canonical and how much is down to Gnome 3. I will have to do some more research. If Gnome 3 is the source of most of the trouble then the outlook is grim indeed. Debian is moving to Gnome 3 which would mean, as far as I am concerned, the end of Debian as well. This is serious because not only is there no other distro base I can turn to but even if there was, it would only be a matter of time before they adopted Gnome 3 also. I suppose all I can hope for is that someone forks Gnome 2 and continues support so that distros can continue to base their UI on Gnome 2. And before anyone asks, "no I will not go KDE".

Ubuntu, netbooks and wireless

I recently got a cheap refurbished Dell netbook, mainly so I can work on documents on the move. I never bothered trying to set up wireless, I plugged in into my lan whenever I needed to download anything, which was not very often. Then Brenda's niece, Emily, said she had a netbook that wouldn't work with wireless. It looked to be a very similar make and model to mine. I offered to help and discovered that it was running Windoze 7. Arrggh. After struggling with that for a few hours I offered to put ubuntu on it for her. I just assumed the wireless would be a breeze. When I got it home I thought "I had better get wireless working on mine first before I trash her Windoze installation" so I had a go. Then I found that getting wireless to work on ubuntu on netbooks can be a right pain. I eventually got mine working. Here's how I did it:

• I downloaded a netbook ISO image for Maverick Meerkat
• I created a bootable USB drive containing the ISO image. I used unetbootin-linux-549, which I built from a source code download.
• I booted the network using the USB stick and did the install.
• I used the lspci command to reveal that the wireless hardware is a RealTek 8176.
• I googled and found someone with the same problem asking for help on a forum. The web page no longer worked but luckily it was still available in the google cache. Some kind person answered with a very detailed step by step set of instructions. I also tried them and they worked for me too!

The instructions are as follows:

• sudo su
• add-apt-repository ppa:lexical/hwe-wireless
• apt-get update
• apt-get install rtl8192ce-dkms,

  This went through a long rigmarole and eventually said

  Building initial module for 2.6.35-24-generic
  
  Done.
  
  r8192ce_pci.ko:
  Running module version sanity check.
   - Original module
     - No original module exists within this kernel
   - Installation
     - Installing to /lib/modules/2.6.35-24-generic/updates/dkms/
  
  depmod.......
  
  DKMS: install Completed.
  

  At this point I rebooted and after the reboot it just worked. Wow.

I am all set to do the same on Emily's machine now but I will leave that to another day.

No-one cares about computers and security.

No-one cares about computers and security.

I care. I am quite interested in computer security from a technical point of view. I also want to keep my data and credentials safe from prying eyes and corruption/deletion, accidental or otherwise and misuse.

However, I now believe that among computer users I am on my own. The only other people that seem to be interested are security consultants. But then they have something to sell. I find this annoying because computer security is important for a number of reasons that really do have the potential to adversely affect me when not done right.

• My money. Is it safe? Do the retail banks take responsibility for making sure that it is not easily stolen and that the account is resistant to fraud? No, they don't. Everyone's bank account is at risk. So are their credit cards and debit cards.
• My personal details. Will they end up in the hands of spammers and junk mailers. Yes, they will. The people that had these details for perfectly legitimate reasons cannot be trusted to look after them responsibly or even with basic competency. This is why everyone is taking to shredding everything these days.
• Impersonation. Will someone be able to log to my online-account, pretend to me, and do all sorts of horrible things in my name? Yes, they can. Newspapers are full of these stories happening more and more often. Do you think your Facebook account is secure? How about your Twitter account? Your PayPal account? Your Amazon account?
• Computer breakins. I want my computer to remain mine and no-one else is allowed in. Thankfully I don't use Microsoft Windows so this greatly increases my chances. However, security holes are discovered in commonly used software every day, including the software that runs on Linux. My own machines are regular attacked using port scanning and password guessing.
• Privacy. I want the stuff that I do on my computers to be private. AFAIK only the UK government have passed a special law that grants them the right to force your decryption keys from you. Also, when using encryption there is the problem that most other people don't know anything about it. It is hard to communicate securely with people that are not interested in keeping the communication secure.

Although I say I care, I have stopped caring in certain areas. This is because I have just got fed up of security not being done right so that it hinders work, even hinders real security and certainly does not provide it. My boss put it so well when he said "security is that which impedes the developer whilst at the same time not offering security".

These areas are listed below.

• o Passwords. I write them down. I keep the number small by reusing them across systems. They are drawn from a small list of passwords that I use over and over. I often don't make them a complex mixture of numbers, letters and special symbols as is recommended. They are often based on words that make them vunerable to a dictionary attack. I keep them all in a notebook in my desk drawer.
• Usernames. I tend to use the same one every time. Yes, it is based on my name.
• I tell people my username and password. Well, at least I do at work. Well at least I would if anyone was interested or asked. The workplace is where security is just a joke so this doesn't matter as far as I am concerned.
• I lend people my security pass. No-one looks at the pass anyway, it is just used to open the door locks. I never wear my pass, I keep it in my pocket.
• I don't lock my terminal when leaving it unattended. The corporate environment tends to force a lock after a very brief period of inactivity so I find myself subconsciously rebelling.
• When I briefly leave my desk I leave the drawer unlocked with the notebook inside containing all my passwords.
• I share my main working directory with everyone indiscrimanently using Windoze Shares and lax permissions. Windoze is not secure anyway, FTP, telnet and CIFS protocols sends usernames and passwords over the wire in plaintext anyway, so what's the problem?
• I don't use two-factor authentication. Well actually I would like to but it would be really useful, like looking after the money in my bank account for example, it is not even offered.
• I don't use public key encryption for sensitive communication. Actually I refuse to use email for sensitive communication. I would like to use public key encryption but usually the person at the other end of the communication does not understand the need and is incapable of using public key encryption so I try to find another way, e.g a face to face meeting or using the telephone (which is not secure either).
• I don't digitally sign anything. After all, no-one uses or understands public key encryption anyway.
• I don't use an Intrusion Detection System (IDS). Actually I did try out an IDS once which is how I know my machines do get attacked. But I never use an IDS at my clients site. They don't use one either. Tough luck. And those systems are typically Windoze-based where a breakin will happen sooner or later.
• I don't use a virus scanner. Well, I use Linux at home, which is a much smaller target for crackers and malware. My clients, being typically Windoze-based, will use whatever the corporate standard is for a virus checker. It is typically configured to not update each day but rather at the behest of the 'security' department. It is often either partially or completely disabled since the continual disk scanning slowers the machine down for developers (word processing users don't notice).
• Whenever I receive a Word or Excel document containing macros I enable them without any consideration of the potential consequences. Such is life in the corporate world. Of course at home I don't use Microsoft Office so it's not a problem there.
• I use Internet Exploiter as my web browser. At home I use Firefox of course. But Firefox has yet to be discovered by the corporate world. IE is still the standard on the desktop and provides a rich source of attack vectors, drive-by-shootings etc.
• I use software that sends usernames and passwords over the wire in plaintext. The most commonly used is FTP. I would like to use secure protocols/commands such as sftp, ssh etc but these have yet to be discovered by the corporate world.

The CUDA compiler can go into a CPU loop

I am working on a project that uses CUDA to perform certain calculations at greater speed, using the machines NVIDIA graphics card. Today I thought I would try to move the project closer to Continuous Integration (CI) using the well-known CI tool, Jenkins (formerly known as Hudson). So I set up a script for Jenkins to run.
Mysteriously, it hung when it got to the bit where it runs the CUDA compiler. I tried building the project interactively in Visual Studio from the Jenkins sandbox. This also hung at the same point. Process Explorer revealed CUDA was consuming roughly 25% CPU but making no progress. It was stuck in a CPU loop. Process Explorer revealed that it was in that part of the compiler that does the preprocessing, so it should have been really fast.
After a lot of tinkering in Jenkins, renaming the project, rebuilding the sandbox, reinstalling CUDA etc etc I eventually fixed the problem. It turns out that CUDA seems to be sensitive to the path that it is running in. If any part of the path contains a directory whose name begins with a dot, then you get the CPU loop. This is unfortunate because Jenkins likes to run in $HOME/.jenkins. When I picked a different Jenkins home (setting JENKINS_HOME via the control panel) and called the directory jenkins rather than .jenkins, then CUDA started working again. Weird, huh? For good measure, I renamed the job so that it did not contain any spaces (Jenkins creates a directory named after the job). I am not sure if this has any effect or not but getting rid of the leading dot definately fixed the problem.

Boost for C++ developers has lost its shine

I have decided that I don't like the C++ boost libraries with just two minor exceptions. The exceptions are the smart pointers that are part of TR1 and boost threads (along with their mutexes and condition variables). They're ok. I haven\'t used all of the rest but I have used the following: --filesystem --named semaphores and named mutexes --sockets (asio) --POSIX date/time I have also looked briefly at the graph library but not actually used it. In the above cases my criticisms are the same:

• I have found several cases where the code does not work. Examples include:
  • filesystem: has difficulty with UNCs
  • named semaphores/mutexes: does not cleanup properly and does not use the underlying WIN32 primitives on MS-Windows.
  • sockets: Counterintuitive model that is contrary to the way that BSD sockets work. For example, async_accept does not do an asynchronous accept!

• The source code is template mania. It is VERY hard to understand. Compare this with Poco or ACE where the code does what you would expect, more-or-less how you would expect.

• The code does not have a clean separation for those parts of the implementation that are platform-dependant. This is very hard to do well but Poco manages to pull it off nicely. Boost fails dismally.

The more I look at boost, the less I like it. It seems to me that it is a prime example of what happens when you give a problem to academics to solve. You get a convoluted solution that only appeals to the Elite and is not very practical. In future, I hope that I get the chance to use Poco for my foundation C++ classes.

NTFS junctions are a poor substitute for symbolic links

Recently, I have been working in a Windoze-XP environment using multiple sandboxes checked out from CVS. Each sandbox needs a large collection of third party libraries (headers, lib and .dll files). Each third party directory takes up around 2GB. This space very quickly mounts up. In Unix we would use symbolic links to make all the third party directories point to a common place. But what to do in Windoze? Some people would say that NTFS junctions is the answer. Well, they can help and I am using them for this purpose but there are some nasties and I am documenting them here. I once saw a signature on USENET which goes: "Those who do not understand Unix are condemned to reinvent it, poorly." This is certainly true of Windoze. It doesn't understand symbolic links so it is no suprise to learn that NTFS junctions are a very poor imitation. Here are several points to bear in mind:

• They are a cross between hard links and symbolic links. Like the hard link, they can onloy refer to files on the same partition, but like symbolic links the path of the link is stored which means it can become invalid. This is because Windoze filesystems do not support links! Also, only directories can be linked.
• Windoze-XP has insufficient support for NTFS junctions despite them first appearing in Windoze-2000. There are no commands to manipulate them. You will have to download and install the SysInternals commands and use the junction command.
• More explicit support for NTFS junctions was added to Vista and is also in Windows 7. But even by these versions of Windoze, support is incomplete. When you use Windows Explorer to navigate to a directory that contains NTFS junctions, if you delete the directory it deletes the files that the junctions refer to!!! See http://shell-shocked.org/article.php?id=284 for the gory details.

The only way to delete a junction safely is to the use the SysInternals junction command. There is no safe way to delete a directory that may contain junctions. Currently, I do not know what the solution is to these problems. The cygwin rm command does not help, it follows the junction references just like Windoze Explorer does. I have a feeling a special delete command may have to be written. I will probably have to write it, in Python, and it will probably use the SysInternals junction command to delete any NTFS junctions it finds.

cron on Windoze using cygwin

There is quite a useful little document, http://csc.csudh.edu/kleyba/cygwin-cron.pdf, that explains how to install the cygwin cron, complete the cygwin setup screenshots. Very nice. But there is a little nasty when setting up cron in a corporate network. Typically the home directory is a network drive which will not be accessible. It took me a while to work out what was wrong and how to fix it. Some of the advice I found whilst googling turned out to be wrong. I tried setting HOME in the crontab. No effect. I tried seting HOME as an environment variable in the control panel. No effect. Finally I edited /etc/passwd and changed the value of the home directory field from /cygdrive/u to /cygdrive/c. Problem solved! Also, note that the pdf document says how to run the cron config command. This was very helpful because other web pages omit this and it turns out to be crucial to getting the service running. Without that config the service gets installed but dies as soon as it starts up.

Setting up PHP and Apache on Windoze

This link is very useful: http://www.thesitewizard.com/php/install-php-5-apache-windows.shtml I mention it here instead of adding it to my list of bookmarks. After all, it's not going to be done every day, is it?

IBM MQSeries queue viewer (Hermes)

At long last I managed to get JMS Hermes working with IBM MQSeries queues. These notes are about how I did it. First, there is a very useful install guide at

http://www.hermesjms.com/confluence/display/HJMS/Installing

A little detail it misses out is that you must set JAVA_HOME as well as HERMES_ROOT. Also, when you track down the jars for MQ, you might find it hard to find the PCF jar. I found it at

 http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24000668&loc=en_US&cs=utf-8〈=en

Finally, there is a flash tutorial at http://www.hermesjms.com/demos/demo_mq.html. It is a little out of date but still very helpful.

Computers and dates

Here follows a very useful link about Julian day numbers: http://www.hermetic.ch/cal_stud/jdn.htm. This explains why julian day numbers apply to a timeframe that has 7980 years. This is often referred to as the Julian period or Scaliger cycle.

The joys of Open Access

I have been clearing out my mailbox of old reference articles, useful links etc and came across this one about Open Access: http://www.earlham.edu/~peters/fos. This is of interest (to me anyway) after my work on Digital Libraries.

Searching PubChem

Here is a useful link for how to search PubChem. http://acdlabs.typepad.com/my_weblog/2008/02/dereplication-v.html. The focus is on having an unknown sample with some NMR data. Well, I thought it was interesting, anyway, especially in the light of my recent work on a Digital Library for a major publisher.

Free news server (NNTP) access

Most UK ISPs have never heard of newsgroups. They think the internet == the web. Duh! So it is very difficult these days to find an NNTP server. This note is to remind me of which one I am using at the moment. Currently it is nntp.aioe.org. I will update this page as this changes over time.Free newsgroup servers do tend to get shutdown eventually (sigh).

getting mySQL set up on ubuntu

It is too painful to setup mysql via a source code build so I used a non-RPM install. It didn't quite work out of the box. Here's what I did: groupadd mysql
useradd -g mysql mysql
cd /usr/local
gunzip < /path/to/mysql-VERSION-OS.tar.gz | tar xvf -
ln -s full-path-to-mysql-VERSION-OS mysql
cd mysql
chown -R mysql .
chgrp -R mysql .
scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --force
chown -R root .
chown -R mysql data
bin/mysqld_safe --user=mysql &
This last step was an attempt to start up the mysql server. It failed, as can be seen by the next command and its output. bin/mysqladmin version
bin/mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
On with an attempt at a fix: touch /var/run/mysqld/mysqld.sock
chown -R mysql /var/run/mysqld/
bin/mysqld_safe --user=mysql &
THIS DIDN'T WORK. Error log said:
[ERROR] Can't find messagefile '/usr/share/english/errmsg.sys'
fixed by saying:
bin/mysqld_safe --user=mysql --language=./share/english &
Now I can connect. All I need to do now is put this into the bootup sequence....

How to rollback a changeset in subversion

Suppose you want to rollback the changes in changeset 2340. You checkout the HEAD into some working directory, then do the following:
svn merge -c -2340 .
svn ci -m
Then you will probably want to set up the working copy so it has the changes you rolled back so you can fix whatever was wrong. Here's the command you use:
svn merge -c -2340 .
This is not very obvious to the beginner. It is done using merge rather than revert. The svn revert command is used to revert uncommitted changes.