Saturday, 8 August 2009

Retrochallenge 2009 Summer Challenge Entry

I participate in retrochallenge - a convenient excuse to utilise retro technology for a month. For the summer 2009 challenge I coded up a fractal generator in VAX Macro-32. You can check out the challenge blog at http://www.wickensonline.co.uk/rc2009sc. I produce this blog on my main website because it is created using ALLIN1 on the VAX by a set of scripts I created for my winter warmup 2009 entry.

One of the images I produced has been used by the competition organiser for the winners mousemat. It can also be ordered in t-shirt format from cafepress.

The above image is also available at 4800x4800 resolution, which shows the recursive nature of the image much more clearly.

Friday, 7 August 2009

Transcoding AVCHD into DVD and Web Formats using the Command Line under Linux

I needed a way of transcoding output from my Panasonic HDC-HS300 into a DVD and web viewable format under linux. There are webpages out there that tell you how to do it, but I had to edit some of the scripts to perform this task automatically from the command line.

I've created two scripts, convert-mts-dvd.sh and convert-mts-web.sh which can be downloaded in the archive convert-mts.tgz. The scripts unpack into a bin directory.

In order to use the scripts you need to ensure you have the latest version of mplayer installed. My version reports:

MPlayer SVN-r29328-4.3 (C) 2000-2009 MPlayer Team

so I would suggest this version or later. The packaged version in the OpenSUSE 11.0 repository is not recent enough - I downloaded the latest source code using subversion and built it locally. The following commands download the latest version, configure and build it to /usr/local. The mplayer binary can then be found under /usr/local/bin so the last command ensures that you pick that version up in preference to a pre-packaged version that might be installed under /usr/bin for example.

svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer
cd mplayer
svn update
./configure --prefix=/usr/local/
make install
export PATH=/usr/local/bin:$PATH
The convert-mts-dvd.sh script is used to convert your HD video into DVD quality, suitable for directly burning to a DVD (for example using my command line tools described in a previous blog post). It contains the following commands:
rm -rf dvd/*
mkdir dvd
for f in *.mts
do
echo $f
mencoder -oac copy -ovc lavc -of mpeg -mpegopts format=dvd -vf scale=720:576,hh
arddup -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=55
000:keyint=15:aspect=16/9:threads=4 $f -ofps 25 -fps 50 -o dvd/`basename $f .mts
`.mpg -demuxer lavf >>`basename $f .mts`.log 2>&1 &
done
wait
Once you have this command on your path, cd to a directory containing the files you wish to convert (which have the extension .mts) and run the command. Note that with the Panasonic Camcorder you can copy the files directly off the camera - you don't need to use the Windows utility provided. It will create a sub-directory called dvd and proceed to convert all the .mts files into DVD format. It creates a log file for each of the converted files giving any output or error messages.

Note the ampersand on the end and the wait command - this script is multi-threaded and will utilise all cores on a mult-core processor. I did some experimentation with versions of this script and found that on a machine with a decent amount of memory the most efficient way of transcoding the videos was to spawn off all the transcode commands at the same time, even when transcoding several tens of mpeg. On my quad-core machine this command completes the process almost exactly four times quicker than processing the command sequentially.

The only disadvantage of spawning all the transcodes at once is that you might impact other processes on the box (I noticed that video streaming would be interrupted for example). You could always use the nice command infront of the call to mencoder to run it at a lower priority, giving preference to existing tasks.

The convert-mts-web.sh provides the appropriate command line arguments for generating a video size 360x288 - it creates a directory called web and places the transcoded video files there.

rm -rf web/*
mkdir web
for f in *.mts
do
echo $f
mencoder -oac copy -ovc lavc -of mpeg -mpegopts format=dvd -vf scale=360:288,harddup -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=1250:vbitrate=625:keyint=15:aspect=16/9:threads=4 $f -ofps 25 -fps 50 -o web/`basename $f .mts`.mpg -demuxer lavf >>`basename $f .mts`.log 2>&1 &
done
wait

Automatic Video DVD Creation under Linux using command line tools

Doing DVD creation properly is a pain. Firstly, you have to master a GUI creation tool. Secondly, every DVD you create seems to involve too much manual work. Wouldn't it be nice if you could just throw a load of mpeg files into a directory and run a couple of commands.

I couldn't find anything out there to fit the bill, so I rolled my own. You can download everything you need in dvd-create.tgz

The prerequisites are the dvdauthor, cdrtools and MPlayer packages. Use your linux distros' package manager to ensure they are installed and up-to-date.

Unpacking dvd-create.tgz gives you a directory dvd-create. Within the directory are three main control scripts: create.sh, burn.sh and play.sh.

Start by copying the mpeg files you want on your DVD into the dvd-create directory. The files will appear on the DVD in alphabetical order, so if required rename (or numerically prefix) them so that the directory listing shows them in the correct order. Then use the create.sh script to process the mpeg files, generate a dvd.xml control script for dvdauthor then run dvdauthor to create the DVD structure within the subdirectory dvd.

#!/bin/bash
rm -f dvd.xml dvd-body.xml
chapter=$(( 0 ))
for f in *.mpg
do
echo " " >> dvd-body.xml
chapter=$(( $chapter+1 ))
done
cat dvd-head.xml dvd-body.xml dvd-foot.xml > dvd.xml
dvdauthor -o dvd -x dvd.xml
The second step is to burn the generated DVD structure to a DVD using the burn.sh script. If you dvd device is not located at /dev/cdrom you will need to edit the script to use the appropriate device.

#!/bin/bash
growisofs -dvd-compat -Z /dev/cdrom -dvd-video ./dvd/

The last step is to test the DVD you've just created using the play script, play.sh. This fires up the mplayer media player to view the contents of the freshly created DVD.

#!/bin/bash
mplayer dvd:// -dvd-device ./dvd

How quick was that!

Monitor your ADSL Connection for Outages & Uptime

I had the misfortune of choosing demon as my ISP and for the first couple of months managed to exceed my fair use policy and experience extended outages. In an effort to document my issues I wrote a couple of scripts on a NetBSD box that monitor the connection and automatically create a graph using gnuplot of the uptime.

The scripts are driven by a crontab script:

$ crontab -l
SHELL=/bin/sh
MAILTO=msw
@hourly /home/msw/router/getstats
5,10,15,20,25,30,35,40,45,50,55 * * * * /home/msw/router/ping-internet.sh
5,10,15,20,25,30,35,40,45,50,55 * * * * /home/msw/router/plot-stats.sh
0,15,30,45 * * * * /home/msw/router/upload-stats.sh

Every five minutes the ping-internet.sh script attempts to ping one of the demon nameservers using its' IP address:

#!/usr/pkg/bin/bash
LOGFILE=/home/msw/router/ping-stats.txt
TIME="`date +%C%y%m%d%H%M`"
/sbin/ping -c 1 158.152.1.58 > /dev/null 2>&1
if [ "$?" == "0" ]
then
STATUS=1
else
STATUS=0
fi
echo "$TIME $STATUS" >> $LOGFILE

This generates a line in the LOGFILE which looks like this:

200907091945 1
200907091950 1
200907091955 1
200907092005 1
200907092010 1
200907092015 1
200907092020 1
200907092025 1
200907092030 1
200907092035 1

where the first number is the date and time and the second number is a '1' if the ping was successful, or a '0' if it was not.

The plot-stats.sh script turns these numbers into a nice graph with the help of GnuPlot then puts the resulting jpeg onto the local apache webserver:

#!/usr/pkg/bin/bash
sleep 30
export PATH=/bin:/sbin:/usr/pkg/bin
export RS=/home/msw/router
export HTDOCS=/usr/pkg/share/httpd/htdocs
export STATS_FILE=$RS/ping-stats.txt
export JPG_FILE=ping-stats.jpg
export SHARE_DIR=/usr/local/archive
export HTTPD_DIR=/usr/pkg/share/httpd/htdocs
cp $STATS_FILE $RS/ping-stats-copy.txt
gnuplot $RS/plot-stats.gnuplot > $SHARE_DIR/$JPG_FILE
cp $SHARE_DIR/$JPG_FILE $HTTPD_DIR

The gnuplot script plot-stats.gnuplot that works the magic:

set terminal jpeg large size 1024, 600
#set terminal dumb
set title "Demon Internet HomeOffice 8000, hostname: waldo"
set lmargin 10
set rmargin 10
set tmargin 5
set bmargin 8
set xdata time
set format x "%d/%m"
set xlabel "Date, 2009" offset 0,-2
set ylabel "ADSL Status"
set yrange [-1:2]
set ytics 0,1,1 ("Off" 0, "On" 1)
set mxtics 6
set timefmt "%Y%m%d%H%M%S"
plot "/home/msw/router/ping-stats-copy.txt" using 1:2 with lines title
"Status at `date`"
#set title "ADSL status" offset 0,-10
# - generated on %Y%m%d %H%M"
show title

Then once every 1/4 hour the upload-stats.sh script pushes the stats up to my remote web host (assuming the connection is up of course!)

#!/usr/pkg/bin/bash
cp /usr/local/archive/ping-stats.jpg /tmp
cd /tmp
ftp ftp://username:password@ftp.wickensonline.co.uk <<-END_OF_INPUT
bin
cd public_html
put ping-stats.jpg
dir
bye
END_OF_INPUT

The end result being this lovely image:

Java/Swing Mirrorbow Remote Control Program

I wrote an application a while ago that I thought would be worth sharing with the world - it's a Java/Swing Application to drive a
Mirrorbow Ethernet IO Interface. I use the interface to control the power to a number of computers and peripherals in my attic, both to make remote operation possible and also to physically isolate them from the mains supply when not in use. I noted a while back that the VAX and Alpha both drew significant power from the mains even when the power supply switch was turned off.

The application looks like this when running:


You simply click on the button to toggle the power. The application mirrors the information displayed in this Swing panel on the built in LCD display of the mirrorbow:


The application is configured using an XML file:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<controller>
<name>mirrorbow</name>
<address>192.168.1.199</address>
<port>
<number>4</number>
<pin>
<number>1</number>
<equipment>
<name>DS10L</name>
</equipment>
</pin>
<pin>
<number>2</number>
<equipment>
<name>DEC3K600</name>
</equipment>
</pin>
<pin>
<number>3</number>
<equipment>
<name>VAX4K90</name>
</equipment>
</pin>
<pin>
<number>4</number>
<equipment>
<name>VAX4K60</name>
</equipment>
</pin>
<pin>
<number>5</number>
<equipment>
<name>EXTSCSI</name>
</equipment>
</pin>
<pin>
<number>6</number>
<equipment>
<name>ZX6000</name>
</equipment>
</pin>
<pin>
<number>7</number>
<equipment>
<name>LINUX</name>
</equipment>
</pin>
<pin>
<number>8</number>
<equipment>
<name>ANCIL</name>
</equipment>
</pin>
</port>
</controller>

I tried to make the implementation as generic as possible. The source code, compiled classes and Windows/Unix command line scripts can be found in mirrorbow.zip

The unit all this controls was hand built using a equipment box, eight IDC connectors, a relay board bought off ebay, a 12 volt power module extracted from a plug in adapter and a mains lead:

Tuesday, 14 July 2009

VAX Macro in the 21st Century

My good friend and work colleague Andrew Beacock posted a recent blog update http://blog.andrewbeacock.com/ with some very nice formatted code which turned out to be a Javascript based syntax highlighter by Alex Gorbatchev which you can find at http://alexgorbatchev.com/wiki/SyntaxHighlighter. Seeing as how I couldn't sleep after about 5:30 this morning, I thought I'd follow Alex's excellent instructions to create a new syntax highlighter for VAX Macro. I'd previously created a VAX Macro mode for jEdit and it was fairly straightforward to cut and paste the opcodes, macros and operands.

The syntax highlighter script file can be found here: http://www.wickensonline.co.uk/scripts/shBrushVAXMacro32.js. Your source code must have open and close angle brackets converted into their html ampersand escaped equivalents. Just add this to your installation of Syntax Highlighter then you can call upon it to format you code, such as in this example.

Friday, 26 June 2009

OpenVMS distros...


I managed to secure a large collection of OpenVMS distribution media from a University Lab who no longer wanted them - a large box arrived with what must have been about 20 boxes of shrink wrapped software - mostly quarterly updates of the Software Product Library, but also including a Compaq branded distribution of OpenVMS itself, and also one branded HP (7.3.2 for VAX and Alpha).

I had sold a few of the SPL packages without really looking in them and the remaining were still shrink wrapped until this evening when curiosity got the better of me. I'm glad I did, as you can see from the pictures the HP and Compaq OpenVMS packages contain a very nice CDROM case. The HP one is branded HP OpenVMS, the Compaq one Compaq OpenVMS.

I also had a copy of Netscape Enterprise Server from 1998 that I opened up, and you can see the contents. It was amazing to see so many 'modern' terms described in a product that is now 10 years old, although I can't say I liked the look of server-side Javascript that much! This box contains Windows NT and Unix versions, including Digital Unix 4.x which I happen to have running on my DEC AXP 3000/600. Possibly too much of a coincidence...