Tester or Noisemaker

Nick
Malik
has an intersting post about Testers and Noisemakers.

 

I like this post because it really hits home for
me.  In my tiny development group, we don’t have any testers. 
Therefore I am the perpetual noisemaker.  I challenge every bad design
(which is most of them).  I challenge every line of source code I see
(which is very few of them).  I drive my team lead crazy.  I try to
make our software “better”.  But in my eyes, no one else sees the benefit
of writing better software and using better designs.  The team has the
unfortunate property of cowboy style development being OK.

In this case I don’t view any such challenges to
the “bad things” as being CLM.  Rather if it does limit my carreer, then I
am better off for it, as this is not an environment in which I wish to
work.  Of course, I am not working at Microsoft like Nick.  Keep
fighting the good fight Nick.  Make Microsoft’s software better!  God
knows there is plent of need and room to do that.

Totally awesomeness for Debian package management.

This was in my ubuntu-devel folder July 4th.  What a
nice holiday gift.

“the latest apt upload (0.6.44.2ubuntu2) to edgy has support for
tracking
of automatically installed dependencies” … “Use “apt-get remove –auto-remove”
to see a list of suggestions what packages are auto-installed and no longer
needed.”

This is VERY awesome.  I get tired of tracking down unknown unneeded
packages.  Especially since Debian and Ubuntu are very good about keeping
packages granular, so often there are many small packages rather than one large
package.  Details on usage is in
the forum
.  Many sysadmins should
rejoice.  Their systems will be cleaner than ever. Much thanks to Michael
Vogt and team.


Software is highway and bridge building, not house building. RE: Fixed Price Bids

Greg has an opinion on Fixed Price Bids over on Code Better.  I agree
with his concerns.  They are definitely important.  Agile
methodologies are all about trust and contracts are all about protection. 
I think Greg is using the wrong analogy and asking the wrong questions.

“an you imagine for a minute going to a builder in a new development around
the corner from your house and signing a $300,000 contract for them build you
a house without a floor plan or other specifications being included in the
contract?”

No, I can’t. But, consider that as detailed as a home
plan is, it is often not followed very closely.  Ask some friends who have
built homes and they will describe where the contractor messed up, or places
were things are different than the plans, sometimes significantly
different.  Further, I propose that the complexity of most software
projects are far more complex than building a house.  Instead I compare it to building a highway and including large
bridges over waterways. 

The highway part is the part of the software that is pretty common and you as
the developer have no worries and can see the outcome already.  The bridge
part is the complex part and the part that you know may have to be modified some
part way into the project.  Medium to Large software applications aren’t
written nearly as often as houses are built.  The process is not as nearly
as refined.  If it were, many of these methodologies which are discussed and being adopted right now would
have long been adopted and there would be no question

The point is, that the contractors who build huge bridges and highways are
often off the mark, and often enough sometimes 40-50% off the mark on
costs.  I’m not sure what the contractual and legal deals look like in
projects of this nature, but my gut tells me that they have more in common with
a large software project than does the construction of a house.


powered by IMHO 1.3

Zip file size limits, faster bzip2, and sliced bread.

Parallel BZIP2 is awesome.  I know bzip2 is a
step up from gzip, but often I wonder if the added CPU cost is worth the extra
compression.  Parallel bzip2 is a parallel implementation which uses
pthreads to achieve near-linear speedup on SMP machines.  It uses libbzip2
underneath so you get full compatibility.

Why would one want this?  Well, I have a quad xeon at work which
takes a very long time to compress certain files which are of 10-20G+
size.  bzip2
precompiled for win32 pegged one CPU for hours on end.  I
figure any speedup would be a good thing.

Why not just use zip?  Well, it turns out that I was doing just that
using infozip and running into a limit with zip files in general.  4GB max file
size.  At first I thought about some workarounds, but I realized that the
best solution was probably to use something without such a limit.

Windows CMD.EXE seems so limiting compared to borne, korne, or bash. 
Yet, there are some gems in there that make simple things simple.

I’m using this command script weekly to zip up some logs.

@echo off
FOR
%%F IN (*_bak) DO c:\sysmaint\bzip2.exe -f %%F

The Parallel Bzip2 page is missing a precompiled win32
version, so I made one using mingw.  I just followed the directions in the
pbzip2 README file, but still, it took a while to download mingw and msys and
get the pthreads library and build bzip2 because I wanted it statically
linked.  So depending on your unix/win-fu using this prebuilt stuff could
save you anywhere from 10 minutes to half a day of time


powered by IMHO 1.3

Spongy? It Is Called JavaSpaces!

Hi David L!

David proposes something called Blanket.NET and “Spongy” Interfaces. This is a great idea and I welcome it as an addition to the dotnet framework, however I think we already have it. A little time with IKVM and JavaSpaces and the same functionality and more should already exist. I’ve been meaning to try the Glitz JavaSpaces implmentation and IKVM, but I have not found time.

David, if you get some time, try out JavaSpaces and IKVM to make it work with .NET.

Awesome reading

I think the last week or so has yielded some of the best software writing I’ve read since I read the book by the same name pieced together by Joel

Most of these are from the blogs at CodeBetter.com.

James Shore writes an interesting aricle on QWAN, Quality With[out] a Name. Which is an intersting response to the c2 article.

Joel wrote a great article on Development Abstraction.

Jeremy Miller wrote an excellent intro to Six Design Patterns

James Bach destroys Best Practices.

On the more meat and potatoes side of things (code not words?)

Jay Kimble gives a great introduction to XSLT called XSLT for the Uninitiated. I really could have used this a few month ago.
Beginning Xpath and Beginning XSLT are the first two of what will hopefully be a continuing series.

Karl Seguin gives great reasons why not to use string concatenation and why to use string.format instead. I’ve taken what he said to heart and I’ll chastize myself when I type + in C# or & in VB.NET.

John Papa wrote about SCOPE_IDENTITY() and @@IDENTITY. It was very timely information for me because I recently had need to use these, and I rarely do SQL work.

Last night at AADND Tim Landgrave shows us the design of a smart client application which uses the CAB. It was very interesting. He mentioned that they converted/upgrade some 30+ “legacy” applications to use their framework, built on top of the CAB. They did so in a very short period of time, saving the company much time and money.

Overall it has been a great week with much thanks to those bloggers and AADND for keeping me fueled.

Use Perl on Solaris to talk to Microsoft SQL Server without root privileges

Use Perl on Solaris to talk to Microsoft SQL Server without root privileges

OR

How to install DBI, DBD::Sybase, and FreeTDS on Solaris without root privileges

1. GCC

A. Download a prebuilt compiler

I grabbed the gcc_small-3.4.2-sol8-sparc-local.gz package from sunfreeware. It has a dependancy on libiconv-1.8-sol8-sparc-local.gz, so grab that too.

gunzip gcc_small-3.4.2-sol8-sparc-local.gz
gunzip libiconv-1.8-sol8-sparc-local.gz

B. Extract the packages

You must be root to use pkgadd, but it turns out pkgtrans will extract a pkg for you.


pkgtrans libiconv-1.8-sol8-sparc-local . SMCiconv # here, SMCiconv is the name of the package. If you don't know, just leave it off the command and you will get a menu
pkgtrans gcc_small-3.4.2-sol8-sparc-local

C. Copy files into place

This extracts the packages to directories with the package name. Within those directories are 2 files and a subdirectory. pkginfo and pkgmap contains some package information. The reloc directory contains the files we want.


cp -ri SMCiconv/reloc/* .
cp -ri SMCiconv/reloc/* .

I like to use the -i command so that I know if files in these packages are overwriting each other. I’m copying them to ., the current directory, you can just as easily copy them to some install path, but I don’t mind keeping a bin, doc, etc, info, lib, libexec, share, include, man in my home directory. Remember we aren’t using any special permissions so /var/tmp or $HOME are probably the only places to which we can write files.

D. Configure the package

So now we have ~/bin/gcc… but take a look, it isn’t executable! It turns out this is what information the pkgmap file holds.


grep "^1 f" SMC*/pkgmap | awk '{print "chmod "$5" "$4}' | sh #set file modes from pkgmap.

Another important step done by pkgadd from the pkgmap is creating symbolic links.


grep "^1 s " SMC*/pkgmap | awk '{print $4}' | awk -F= '{print "ln -s "$2" "$1}' | sh

2. Get a development environment

From this point on, I like to have a develop environment defined in my shell. I source a shell script which looks like this.


PREFIX=$HOME
PATH=$PREFIX/bin:/usr/ccs/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/sfw/bin:/usr/local/bin:/usr/local/sbin:/usr/openwin/bin
PERLLIB=$PREFIX/lib/perl
LD_LIBRARY_PATH=$PREFIX/lib
MANPATH=$PREFIX/man:/usr/share/man:/usr/man:/usr/local/man:/usr/sfw/man
PERLLIB=$PREFIX/lib/perl:$PREFIX/lib/perl/sun4-solaris-64int
export PATH
export LD_LIBRARY_PATH
export MANPATH
export PERLLIB

3. Freetds

If you have ever built source packages which use the autotools system, this is straightforward. Just be sure to set your PREFIX.

Download freetds from www.freetds.org.


gunzip -dc freetds-stable.tgz | tar -xf - # Extract the tarball
pushd freetds-0.63 ; ./configure --prefix=$HOME && make && make install ; popd

4. DBI

Download DBI-1.50.tar.gz from dbi.perl.org. It depends on Test-Simple, so get that from CPAN.


gunzip -dc Test-Simple-0.62.tar.gz | tar -xf -
pushd Test-Simple-0.62
perl Makefile.PL PREFIX=$HOME/src INSTALLSITELIB=$HOME/lib/perl && make install
popd
gunzip -dc DBI-1.50.tar.gz | tar -xvf -
pushd DBI-1.50
PREFIX=$HOME/src CC=gcc LD=gcc PERLLIB=$PERLLIB perl Makefile.PL
perl -pi -e "s/ cc$/ gcc/;s/-xO3 -xdepend/-O3/;s/ -KPIC/ -fPIC/;s@INSTALLSITELIB = .*\$@INSTALLSITELIB = $HOME/lib/perl@;s@INSTALLSITEARCH = .*\$@INSTALLSITEARCH = $HOME/lib/perl/sun4-solaris-64bit@;s@PREFIX = .*@PREFIX = $HOME@" Makefile #massage solaris's stupid perl.
make install
popd

Using Sun’s perl to build perl modules can be a challenge, but a few touchups to the Makefile allow us to accomplish what we want.

5. DBD::Sybase

CPAN docs are excellent: http://search.cpan.org/~mewp/DBD-Sybase-1.07/Sybase.pm

Download DBD::Sybase http://www.peppler.org/downloads/DBD-Sybase-1.07.tar.gz


gunzip -dc DBD-Sybase-1.07.tar.gz | tar -xf -
pushd DBD-Sybase-1.07
perl Makefile.PL PREFIX=$HOME/src INSTALLSITELIB=$HOME/lib/perl CC=gcc LD=gcc # press enter at all the prompts.
perl -pi -e "s/ cc$/ gcc/;s/-xO3 -xdepend/-O3/;s/ -KPIC/ -fPIC/;" Makefile #massage solaris's stupid perl.
make install

6. Use it!

Using it involves configuring your etc/freetds.conf file and writing some perl.

The only special part of this is using a sunfreeware package without root access. This opens up many exellent possibilities for using precompiled packages. Getting a compiler opens up even more, and you don’t even have to nag your system administrator. Of course system administrators probably hate developers like me who used to be system administrators.

trivial example usage:


#!/usr/bin/perl -w
use strict;

use DBI;

$ENV{DSQUERY}="myserver"; #this is a [server] defined in freetds.conf

print 'Enter your username: ';
chomp (my $username = <>) ;
print 'Enter your password: ';
`stty -echo`;
my $password = <>;
chomp $password;
`stty echo`;

my $dbh = DBI->connect('dbi:Sybase:database=mydb', $username, $password) or die DBI->errstr;

my $sth = $dbh->prepare('select * from Table');

$sth->execute;

while ( my @row = $sth->fetchrow_array ) {
for my $c (@row) {
if ( ! defined $c ) {
$c="";
}
}

print join(",",@row)."\n";
}
$sth->finish;
$dbh->disconnect;

RE: Absolute Beginners – PHP 101 (part 8): Databases and Other Animals

A very good friend of my is a web designer who has always struggled with programming. He found the Absolute Beginners – PHP 101 series of articles from Zend to be extremely helpful. I noticed he is learning things better this time simply by the types of questions he asks me. He started asking me some database things and my immediate response was “don’t do it that way”. My response was of course wrong. After all, he was just learning. The 8th article in the series introduces database access and honestly it feels a bit out of place after the first 7 articles. The first 7 take an almost programming 101 (php101 right?) approach and give good fundamentals. With a little bit more background in each article, the series could almost be the first few weeks of an introductory college level computer science (programming) course. The next steps would be to introduce data abstraction, common abstract data types, and algorithms, but Zend knows their audience here. These are people who just want to get things done.

http://www.zend.com/php/beginners/php101-8.php is the article to which I am refering. It is fine and there is nothing wrong with it, but it does lead the new programmer down a dangerous path. There is no mention of abstraction.

Using an abstract library instead of direct database function calls means that if you want to switch databases from MySQL to PostgreSQL in the future, it will be a lot easier. Here is what the example code might look like.

Instead of calling the mysql_ suite of functions directly, PHP provides an excellent Database Abstraction library in the PEAR library known as PEAR::DB. I used PEAR::DB back in 2001 on one medium sized web project and it worked great. I had used it prior for many months without issue. I’m sure it has matured over the past 5 years.

I had initially written an article directly converting the examples in the Absolute Beginners – PHP 101 article 8 to using the PEAR libraries. Due to a horrible typo, I lost an hour of work and I never revisited this task. Instead of write a side by side comparison of direct mysql_ function calls to PEAR::DB connection instance OO calls, I’ll simply reference a PEAR:DB tutorial article. A great article is available at evolt called Abstract PHP’s database code with PEAR::DB. A huge list of resource is available from PHP Kitchen. International PHP magazine has an Introduction to PEAR which eases the reader into PEAR::DB.

So before calling a mysql_, pg_, ibase_, fbsql_, db2_, maxdb_, mssql_, sqlite_ or some other database function, consider using PEAR::DB, so that you don’t have to guess if s/mysql_/pg_/ will work or not. Of course, writing cross database SQL is something else entirely, but this is a necessary first step.

Things I learned about Python

From Mark Ramm’s in person class. Only 11 people, including myself, so I didn’t feel like I was squatting. Well, at least not too much.

My knowledge from Python 1.4 days is pretty much useless.

Mostly in comparison to other languages.

Lists in perl vs. python:
Python Perl
l.expand(element) push @l, $element
l.pop() pop @l
l.insert(0,element) unshift @l, $element
l.pop(0) shift @l

help() is every bit as capable as perldoc

List comprehensions are just a big name for some iterator syntax sugar. (sorry python guys, that is my take).

Introspection in python is WAY easier than Reflection in .NET.

help(modules), help(), import __builtin__;help(__builtin__), are great starting points for help in python.

True, False, and Null have some interesting properties. 0, [], (), “” (zero, empty list, empty tuble, empty string) are all considered false. Yet comparison with True or False still yeilds false. I think the logic is similar to NULL in SQL, but I have not found definitive Python docs stating such.

Python uses different terms to describe things. Member variables are called attributes. C, C++ and C# have attributes but they are entirely different things.

There are no member access modifiers in python (no private, public, protected and internal).

This TurboGears course is definitely beyond me. The fact that I’m having trouble installing it on my laptop is a bummer. I’m thinking setuptools and windows don’t like each other that much. TurboGears subversion may work better, I don’t know. (5 min later… I do know. I hate that IBM ships python and it is in my default path on this T42… Major Oops!)

TurboBlog is a turbogears open source blog software which may be good to examine when learning turbogears.