Earlier in the year, I wrote a similar article discussing the Catalyst Web Framework and the MojoMojo Wiki software. At the beginning of December 2009, I wrote an article which was published in the Catalyst Advent Calendar. I’m re-posting it here for posterity, and because it is still relevant to others today.

–

Introduction

My second “lab” for ECE4464: Power Systems II studied the effects of distributed generation (in particular, a large-scale wind turbine generation project) on the power system. Like ECE3333 (Power Systems I), this course is being taught by Prof. Rajiv Varma, Ph.D.

Using PowerWorld’s Simulator software, we connected a large four-reactor nuclear generation plant (750MW per reactor, constant output) via two parallel transmission lines with a large city (modeled as an infinite bus). At the midline between the nuclear reactors and the large city, a transformer is installed at the midline bus to supply a different voltage to a nearby town. This town is near the proposed connection point of the Wind Turbine Generation (WTG) project; this project has a peak output of 85MW.

From the objectives:

In this lab, our objective is to study the potential impact of integration of distributed generation (in particular, a wind farm) using the PowerWorld Simulator software.  We simulate a very large capacity plant (a nuclear plant consisting of four reactors producing 750 MW each).  Because these reactors are large, they will provide some voltage regulation by supplying or consuming MVARs.

The nuclear plant serves a large city via two parallel transmission circuits, as well as the smaller city at the mid-line of one of these lines.  To simplify this lab, we model the large city as an infinite bus, which consumes any excess power generated by the plant.

In this manner, we can explore various phenomena resulting from distributed generation systems like wind farms, including the effects on power transfer and power system stability.  This lab provides insight on two very important issues in power systems, notably, the addition of distributed generation and the challenges involved with electrifying remote communities.

For my full report, see: Power Systems 4464 Lab 2 (PDF). Note that the small town bus has a constant 20Mvar reactive power demand, with a 40Mvar (nominal) shunt capacitor installed initially. Some modifications are made to the compensation scheme as part of the study and report.

In the study of electrical engineering, power factor comes up quite often in terms of its various mathematical definitions, but people seem to overlook its real-world relevance. Though there are some regulations governing power factor, the way residential users are billed for electricity often leaves us in blissful ignorance of the importance of power factor. In fact, power factor is a measure of efficiency.

Starting from first principles, let’s look at the equation for instantaneous power in electrical systems:

Of course, there are similar definitions for mechanical power (which involve torque and speed rather than voltage and current as above).

In the best case, the voltage and current waveforms will be identical, which means that they are both sinusoidal with crests and troughs occurring together. For passive devices like light bulbs (which are purely resistive loads), this is exactly what happens. However, some devices (such as capacitors and inductors) store energy for a short period of time, causing the waveform of the current to be phase shifted or displaced, relative to the voltage wave.

If we use the “average” values of voltage and current, we can determine what is known as apparent power. Even though I call them average, what we really use are the “root-mean-square” values — the reasons we use this measure are beyond the scope of this article, but suffice it to say that we can’t use a normal average for sine since it would simply be zero. For you statisticians out there, RMS is related to standard deviation (it’s a special case where your mean is zero).

Though its units are identical to those of Power (Watts), we use a different unit convention for this value, the Apparent Power (Volt-Amps, or VA).

Power factor is simply the ratio of real power compared to apparent power:

For linear devices which do not store energy, real power and apparent power are the same, so the power factor is 1 (sometimes people call this “unity power factor”).

If, however, an energy storage device like an inductor or capacitor stores energy and simply return it back to the source, then the power factor will be reduced (since power is being transmitted over the line, stored temporarily and then sent back). Only real power contributes to work actually done — whether it be heating your room or turning a motor.

As mentioned earlier, inductors and capacitors cause the voltage and current to be shifted relative to each other. This results in what is called the Displacement Power Factor (the angle, φ, refers to the angular displacement between voltage and current):

However, as we are moving forward in semiconductor techologies, we are increasingly encountering more and more nonlinear devices which introduce something known as Harmonic Distortion. Basically, it makes the current waveform “noisy” compared to the voltage reference; usually this is because a device switches on and off (goes from periods of drawing some current to zero current) instead of merely being proportional to applied voltage.

Total harmonic distortion (THD) is a measure of how “noisy” the current waveform is; for example, if you draw lots of power in short-duration bursts, the current wave won’t look like a sinusoidal waveform at all. This is also known as a shape factor since it will be 1 (unity) for perfectly sinusoidal current, and smaller otherwise.

THD is measured as a percentage, so its value is somewhere between 0 and 1.

The overall power factor takes both distortion (current waveform shape) and displacement (current waveform phase difference, relative to the voltage) into account:

So now we have identified an equation useful for determining the overall power factor of your equipment, especially for things like computers, cooking, heating, washing/drying of clothes, etc. But what does power factor have to do with efficiency?

To answer that question, let’s rearrange our power factor vs (real and apparent) power equation to look at what happens to current. We’ll be looking at RMS current, which is important because it is one of the primary factors that determines maximum loading of a given power line.

Since the amount of power we’ll need and the RMS voltage (line voltage) are effectively fixed, we can see that power factor and RMS current are inversely related. That is, with a lower power factor, we will require higher RMS current to deliver the same amount of power to our load; our RMS current is lowest when the power factor is 1 (unity).

Since power dissipated across a transmission line is:

There are finite limits on the amount of current that can be transmitted (greater power dissipated results in more heating of the wires, which can cause them to expand significantly or to melt), a unity power factor means a more effectively utilized infrastructure.

If everything we connect to the power system has a low power factor, it will result in an inefficient use of existing infrastructure (since we are transmitting more current than necessary, which also increases total line losses). It also means we will need a greater investment in infrastructure sooner, which is a challenging issue facing electric power utilities responsible for distribution of power.

I’ve recently been pushing for greater support for Catalyst and MojoMojo on Debian. For the uninitiated, Catalyst is a Model-View-Controller Framework designed for writing web applications. MojoMojo is a Wiki application based on Catalyst that provides a lot of neat features; while it seems less popular than Wikimedia’s MediaWiki software, it’s still got plenty of features other wikis don’t.

Here’s a blurb about it from their homepage:

We also have a bunch of features you won’t find in every wiki, like an attachment system that automatically makes a web gallery of your photos, live AJAX previews as you are editing your text, and a proper full text search engine built straight into the software.

Unfortunately, such a rich feature set comes at a price — this shiny piece of software has a rather large dependency chain. As a result, building the module (after building its prerequisites) from CPAN is both slow and prone to failure, since each module must be individually retrieved, extracted, built, tested and then installed.

To make matters worse, any failure anywhere in the chain (perhaps a new version of a module breaks things) will cause a complete failure to build the module — either Catalyst or MojoMojo — which has some serious implications for production applications.

In Debian, we mitigate this risk by having separate unstable and testing distributions, so if a newer version breaks things in unstable, we will catch it and have a chance to fix it before the package makes it into testing. By packaging these modules for Debian, we get the advantages of a faster installation process (since we’re installing pre-built binaries) combined with better Quality Assurance.

One of the big issues blocking both of these has been missing copyright information for a lot of modules. I’ve worked a lot with Matt S. Trout, one of the primary people behind coordinating the efforts of the Catalyst project, and gathered the necessary information for an upgrade and upload into Debian.

Recently, libcatalyst-modules-perl (version 35) and libcatalyst-modules-extra-perl (version 4) were uploaded to Debian, containing many necessary updates and fixes to improve the Catalyst experience on Debian. The next big push is to get MojoMojo’s dependencies packaged (currently only String::Diff is blocking it, due to missing copyright information).

A bounty of $150 is being offered by one of the MojoMojo developers to the first person who can re-implement the String::Diff functionality in a free/open source way.

Thyristor-Controlled Series Compensation (TCSC) is used in power systems to dynamically control the reactance of a transmission line in order to provide sufficient load compensation.  The benefits of TCSC are seen in its ability to control the amount of compensation of a transmission line, and in its ability to operate in different modes. These traits are very desirable  since loads are constantly changing and cannot always be predicted.

TCSC designs operate in the same way as Fixed Series Compensation, but provide variable control of the reactance absorbed by the capacitor device. The basic structure of a TCSC can be seen below:

A thyristor-controlled series compensator is composed of a series capacitance which has a parallel branch including a thyristor-controlled reactor.

TCSC operates in different modes depending on when the thyristors for the inductive branch are triggered.  The modes of operation are as listed:

• Blocking mode: Thyristor valve is always  off, opening inductive branch, and effectively causing the TCSC to operate as FSC
• Bypass mode:  Thyristor valve is always on, causing TCSC to operate as capacitor and inductor in parallel, reducing current through TCSC
• Capacitive boost mode: Forward voltage thyristor valve is triggered slightly before capacitor voltage crosses zero to allow current to flow through inductive branch, adding to capacitive current. This effectively increases the observed capacitance of the TCSC without requiring a larger capacitor within the TCSC.

Because of TCSC allowing different operating modes depending on system requirements, TCSC is desired for several reasons. In addition to all of the benefits of FSC, TCSC allows for increased compensation simply by using a different mode of operation, as well as limitation of line current in the event of a fault. A benefit of using TCSC is the damping of sub synchronous resonance
caused by torsional oscillations and inter-area oscillations.  The ability to dampen these oscillations is due to the control system controlling the compensator. This results in the ability to transfer more power, and the possibility of connecting the power systems of several areas over
long distances.

–

This article was taken from the introduction of a report which was written by a partner and I, submitted to ECE3333: Power Systems I, taught by Professor Rajiv Varma at the University of Western Ontario.

The main purpose of series compensation in power systems is to decrease the reactive impedance of the transmission line to reduce voltage drop over long distances and to reduce the Ferranti effect.  By adding series capacitors to the line, engineers can compensate for the physical inductance inherent in the transmission line.  The voltage drop across the line is reduced with more compensation, allowing more power to be received by the load for any given sending power.  Two main types of series compensation are fixed series compensation, and thyristor controlled series compensation, each with their own advantages.

Fixed Series Compensation

Fixed series compensation (FSC) of a line is desirable for power transmission due to the effects of line reactance modification.  By adding series capacitance, the reactive impedance of the line decreases, thus lowering the voltage drop across the transmission line.  This effect can be seen through the simplified power flow equation (see the post about Power Transfer) obtained by neglecting line resistance and line charging capacitance.

Line reactance is counteracted by a series capacitance, resulting in overall lower line impedance and a lower voltage drop across the line.

By adding the series capacitance, it can be seen that the receiving line end voltage will be closer to the sending line end voltage.  This decrease in voltage drop across the line allows more power to be transferred over the line for any given sending line end voltage.

The advantage to using FSC compared to thyristor controlled series compensation is price.  Usually FSC allows for a majority of compensation for a lower cost when compared to thyristor controlled series compensation. The following phasor diagram demonstrates the effect of series compensation:

–

This article was taken from the introduction of a report which was written by a partner and I, submitted to ECE3333: Power Systems I, taught by Professor Rajiv Varma at the University of Western Ontario.

Recently, I completed my first “lab” for ECE4464: Power Systems II. Like ECE3333 (Power Systems I), this course is being taught by one of the most inspiring professors I have ever had, Prof. Rajiv Varma, Ph.D.

Using PowerWorld’s Simulator software, we repeated one of our basic labs from ECE3333 as an introduction to computerized modelling of power systems. We connected a single synchronous machine to an infinite bus across a 600km, 1000MW-SIL power line.

It is simplest for me to just lift the objectives from my lab report:

In this lab, our objective is to simulate a simple single machine infinite-bus configuration using the PowerWorld Simulator software.  We design a local generator system (a synchronous generator) having a nominal generation capacity of 500MW and with no predefined peak generation (that is, the generator is modelled as having infinite generation capability).

In this manner, we can explore various phenomena like power transfer, power system stability and the effect of shunt compensation on the midline.  We model a 600km span of transmission line with a shunt compensation device installed at the midline (300km from both ends) and determine the stability limit with and without this compensation device enabled.

Please see the following images, which show the simulation being run in PowerWorld:

500 MegaWatt generation, no compensation

500 MegaWatt generation with Synchronous Condenser Compensation

For my full report, see: Power Systems 4464 Lab 1 (PDF). Note that the synchronous condenser installed at the midline is a Switched Shunt Compensation unit. I thought the standard inductor/capacitor schematic symbol looked a little boring, so I overlaid a synchronous condenser on top of it.

The development of high-power thyristors and Insulated Gate Bipolar Transistors (IGBT) enabled the cost-effective provision of FACTS devices.  The actual behaviour of these devices is beyond the scope of this article, but on a basic conceptual level, they are simply fast acting switches, controlled by some external means (a trigger).  Triggers can be either electric (a voltage applied at the gate terminal) or photonic (light), the latter of which is useful to isolate the control system electrically.

Thyristors can be conceptualized (and indeed, are drawn on schematic diagrams) as diodes with a switch (the gate voltage or photoelectric stimulus). In FACTS correction systems, whereby a thyristor should act as essentially a fast-acting switch, power needs to be transferred in both directions. Thyristors are also used in high-power rectifier circuits as well, particularly for High-Voltage Direct Current (HVDC) transmission.

How much faster is thyristor-based switching compared to mechanically-switched circuit breakers? Because thyristors are semiconductor devices, they can switch on the order of milliseconds. Conventional circuit breakers, on the other hand, take much longer to switch. They can switch in one or two cycles (of the 50-60Hz mains frequency), though for power system protection purposes, this is considered a rather slow switching speed.

The graph below illustrates the difference:

Additionally, mechanically-switched capacitors do not have sufficient switching speed to support extremely rapid switching nor can they be realistically switched more than a few times per day.

We can see that advanced communications and control systems play an important role in flexible transmission and distribution systems.

–

This article was taken from the introduction of a report which was written by a partner and I, submitted to ECE3333: Power Systems I, taught by Professor Rajiv Varma at the University of Western Ontario.

One of the most often overlooked–yet arguably most important–issues in software development is copyright and licensing of works. In particular, I will discuss how this affects the open source software community with relevance to the Debian project.

As with any artistic or creative works, software is protected by copyright and its use is often governed by some sort of license. Please note that I am not a lawyer and I am not qualified to give legal advice, so take my suggestions with a grain of salt and please do leave a comment if you know something that I don’t.

A license is a legal contract that permits end users use of software under agreed-upon guidelines. In the open source community, licenses protect the integrity of free software by ensuring that they continue to remain freely available. For example, the GNU General Public License (GPL) stipulates that any derivative works of GPL-licensed code must distribute source code back to the community, which enables a two-way sharing of information between the originating software developers and the others who benefit from their work. Other licenses, such as the BSD License, are more liberal and do not have this restriction, but do have a disclaimer of warranties which shields authors from unintended legal consequences of their work.

Though licensing is probably the most important document detailing the relationship of the supplier (software developer or team) and other users, it cannot mean anything without copyright. In general, it is most useful to provide a copyright statement somewhere in resulting packages. A copyright statement is what allows authors to assert a particular license in the first place.

Moreover, license terms can only be changed when all copyright holders agree to the change. Unless you are explicit in your copyright conditions in the beginning, this can lock your project in to an undesirable license.

To make matters even more complicated, the Berne Convention for the Protection of Literary and Artistic Works (or simply Berne Convention as it’s most often called) describes a mechanism by which copyright is automatically in force upon creation of a work, even if the author does not explicitly assert it. For software, this effectively means that anyone who contributes any code is automatically the copyright holder on their contribution, which means that things quickly get complicated when there are many authors and contributors involved.

In Debian, we cannot and do not distribute software without knowing copyright information (including years of copyright, names, e-mail addresses where people can be reached, or a web site in the case of an incorporated entity). This is pursuant to the Debian Free Software Guidelines (DFSG), which require that we distribute only “free” software in our main repository–it’s part of our Social Contract.

In this regard, I would make the following recommendations:

1. When beginning any project (open source or not), include a copyright statement immediately. It will eventually become a force of habit; this is a very good thing, and will pay dividends in the future.
2. Establish a policy whereby contributors are asked to assign you copyright of their work; make a note of this somewhere in your documentation. Better yet, if you are part of an incorporated entity, assign copyright to that entity.
3. Be explicit about your licensing terms and make sure to include copies of the license with your software. This helps to resolve ambiguities where there are several derivatives of a license (occasionally, developers license software under the BSD License without specifying which version they mean)
4. Be wary of the “Public Domain” — this is an even more contentious issue than choosing an appropriate license. It is probably preferable to use a non-restrictive license such as the aforementioned BSD License (and its variants) or the MIT/X11 license, which is even more permissive.

The power flowing across an arbitrary power transmission line depends on the sending end voltage (V₁), receiving end voltage (V₂), line impedance (Z = R + jX) and the phase difference between the sending and receiving terminals.  Since the line resistance (R) is usually very small compared to its reactance (X), that term is negligible in the Power Transfer Equation (Equation 1, below).

The second equation indicates that the power factor (the cosine of the angle between current and voltage) has a direct impact on total power transfer. Power factor is somewhat like a measure of efficiency: it’s the ratio of real power delivered to your circuit compared to “apparent power”, which is temporarily absorbed by reactive elements like capacitors and inductors (also called reactors) and then returned to the generator.  This is because these are energy storage devices.

Lastly, equation 3 indicates the amount of thermal power lost dissipated to the resistance of the power line. It is converted to heat at a rate proportional to the apparent power flow, which is why a poor power factor is particularly bad for the grid — it means more power is lost due to heat in the transmission line. Since current results in heating of the line, excessive current will cause the line to physically droop. It becomes dangerous because it increases the likelihood that the line will contact another phase (line-to-line fault) or ground (via a tree or house, for example).

As a result, power factor (and subsequently, apparent power flow) affect the power system’s economic viability and profitability.

–

This article was taken from a report which I co-authored. It was submitted to ECE3333: Power Systems I, taught by Professor Rajiv Varma at the University of Western Ontario in Spring 2009.

The acronym “FACTS” is a blanket term used to describe a family of power electronic devices that can improve power quality and transmission capability of existing power transmission systems, without significant investment in infrastructure.  Many of the most important issues in power engineering are simple to understand but complex to address, particularly since the system operates on a much larger scale than the simplified models studied in the classroom.

While initial investment in FACTS devices can number in the millions of US dollars due to their sheer size, the payback time is usually short due to the cost savings they can provide.  For the utility, installation of FACTS devices to increase utilization of existing transmission line assets is preferable to the planning and deployment of additional lines, since the cost of building a new transmission line can be in the range of millions of dollars per kilometre and take several years to complete.  Power utilities and consumers can realize the benefits of FACTS much more quickly, since planning through to deployment and testing only takes about a year.

In essence, FACTS devices can increase the efficiency of transmission lines up to their thermal limit, which can increase maximum power transfer from 50% to 100%.  Evidently, the time and cost benefits are substantial, however, once transmission lines reach their thermal limit, new lines must be constructed.  FACTS devices cannot offer any benefit once a line is operating at maximum efficiency and has reached the thermal limit.

–

This article was taken from a report which I co-authored. It was submitted to ECE3333: Power Systems I, taught by Professor Rajiv Varma at the University of Western Ontario in Spring 2009.

Okay, so this is a long-awaited follow-up to my first post on the topic of  Debian Perl Packaging. Some of you might note I was pretty extreme in the first post, which is partially because people only really ever respond to extremes when they’re new to things. When you first begin programming, the advice you get is “hey, never use goto statements” — but as your progress in your ability and your understanding of how it works, what it’s actually doing in the compiler — then it might not be so bad after all. In fact, I hear the Linux kernel uses it extensively to provide Exceptions in C. The Wikipedia page on exception handling in various languages shows how to implement exceptions in C using setjmp/longjmp (which is essentially a goto statement). But I digress.

Back to the main point of this writeup. Previously I couldn’t really think of cases where packaging your own modules is really all that useful, especially when packaging them for Debian means that you benefit many communities — Debian, Ubuntu, and all of the distributions that are based on those.

During a discussion with Hans Dieter Piercey after his article providing a nice comparison between dh-make-perl and cpan2dist. (Aside: I feel like he was slightly biased toward cpan2dist in his writeup, but I’m myself biased toward dh-make-perl, so he might be right, even though I won’t admit it.)

I’m really glad for that article and the ensuing dialog, because it really got people talking about what they use Debian Perl packages for, and where it is useful to make your own.

Firstly, if you’ve got an application that depends on some Perl module that isn’t managed by Debian, but you need it yesterday, then you can either install that module via CPAN or roll your own Debian package. The idea here is to make and install the package so you can use it, but also file a Request For Package bug at the same time — see the reportbug command in Debian, or use LaunchPad if you’re on Ubuntu. This way, when the package is officially released and supported, you can move to that instead, and thus get the benefits of automatic upgrades of those packages.

Secondly, if you’ve got an application that depends on some internally-developed modules, then they probably wouldn’t exist on CPAN (some call this Perl code part of the DarkPAN), except in the rare case that a company open sources their work. But corporations will never open source all of their work, even if they consider the implications of providing some of it to the open source community, so at some point or another you’ll need to deal with internal packages. Previously, the best way to handle this was to construct your own CPAN local mirror, and have other machines install and upgrade from it — thus your internal code is easily distributed via the usual mechanism.

One of the advantages of using CPAN to distribute things is that it’s available on most platforms, builds things and runs tests automatically on many platforms. CPANPLUS will even let you remove packages, which was one of the main reasons I am so pro-Debian packages anyway. However, it does mean you’ll need to rebuild the package on other systems, which is prone to failures that cost time and money to track down and fix. CPAN and CPANPLUS are the Perl tradition of distributing packages.

If you are using an environment mostly with Debian systems, however, you may benefit from using a local Debian repository. This way, you only need to upgrade packages in your repository, and they’ll be automatically upgraded along with the rest of your operating system (you do run update and upgrade periodically right?). There is even the fantastic aptcron program to automate this, so there’s really no excuse not to automatically update.

In either case, creating a local package means you will be able to easily remove anything you no longer need via the normal package management tools. You can also distribute the binary packages between machines — though it sometimes depends on the platform (for modules that incorporate C or other platform-specific code that needs to be rebuilt). Generally, most Perl modules are Pure Perl, and thus you can compile and test it once, on one machine, and distribute it to other ones simply by installing the .deb package on other machines. You can copy packages to machines and use dpkg to install them, or better yet, create a local Debian mirror so it’s done automatically and via the usual mechanism (aptitude, etc.)

In conclusion: if you’re going to make your own Debian packages, do so with caution, and be aware of all the consequences (positive and negative) of what you’re doing. As always, a real understanding of everything is necessary.

It’s been some time since I re-installed Debian over my Kubuntu install, so I thought I’d discuss some reasons why I changed back to Debian, what my experience was like, and some learning opportunities.

One reason I made the switch was because there was a utility newly packaged for Debian, Frama-C, which was not available in Kubuntu at the time. It also frustrated me that I was having various frustrations with the installation, not the least of which was an unreliable and quite crashy KDE Plasma.

When I reinstalled this time, I picked the normal install but told it to install a graphical environment, which gave me a GNOME desktop environment. I actually rather like it, the setup didn’t ask too many questions and everything was set up perfectly. There was some minor tweaking, but it was all done by the easily accessible System menu and all the applets therein.

Now, I wanted to be able to use the server both as a virtual machine and as a physical dual-boot. This wasn’t working properly with GRUB-2, so I had to stay with version 1.96, which works rather well. I even spent some time making a pretty splashimage for it, which looks rather nice, even if I don’t see it all that often.

If I boot into the Virtual Machine, all the hardware is detected properly, and there aren’t even complaints about the fact that a bunch of hardware disappeared — certainly very good news if you decide to do something like move your hard drive to a different machine. Likewise, if I boot into the desktop, everything works well there too.

One issue I came across during the installation was having to teach Network-Manager how to configure my network interfaces. In my VMware NAT setup, there is no DHCP server, so the IP address, subnet and gateway information needs to be statically defined. Luckily, Network-Manager was able to do this based on the MAC address of the adapter — inside my virtual machine, it had a VMware-assigned static one. Through this, Network-Manager had an easy way to determine how to configure my network, and it works beautifully for Ethernet and Wireless (when Debian is running as the main operating system) and also for VMware NAT (when inside the virtual machine container).

Anyway, I have now been developing quite happily inside a Debian + GNOME desktop environment. The system runs fine even within a constrained environment, though I miss KDE’s setup with sudo; with GNOME, the option seems to be to have the root password entered every time privilege escalation is necessary. I don’t like using a root password — on my server system I don’t use the root password at all, and do everything I need to do via sudo. It’s okay for me because I log into the server with a private key and have disabled SSH password authentication for security reasons.

One thing that is still weird for me is that my system currently shows a time of 01:53 even though it is 23:57 in my timezone. Presumably the few minutes of difference is because the virtual machine clock and my system hardware clock aren’t synchronized perfectly, but more than that, I think it’s an issue with the Date applet somehow. I haven’t looked into this because the thing is running inside a virtual machine, so it doesn’t bother me much.

I have looked high and low to see where to change the time zone, and to my knowledge the system knows that it’s in the America/Toronto time zone. The house picture next to Timmins (the city I am in right now, though it doesn’t matter since the timezone is the same) seems to indicate to me that it’s set to the appropriate time zone.

I think it’s due to VMware synchronizing the virtual machine clock with my host machine clock. Windows (my host operating system) stores the time in the local format, which I believe Linux thinks is UTC. Still, it doesn’t explain the weird display it’s got going.

Someone noted last time that I didn’t make direct mention of which programs are only offered on Windows and not on Linux/etc, and that do not have reasonable replacement on these systems. Kapil Hari Paranjape noted that I was sounding somewhat like a troll by simply saying that I don’t think Linux is yet ready to replace my environment. Here was my reply:

Far from a troll, I’d really like Debian and Ubuntu, but moreso Linux in general, to improve at the pace it has been doing so. It’s made great progress since the last time I tried it out on my desktop, but I have to acknowledge that there are lots of rough edges right now that should be worked out.

One of the advantages of huge proprietary development organizations like Microsoft is that they have tons of developers and can implement new features at a relatively quick pace, even if they’re half-assed. Developers’ pride in the FOSS community prevents this overly quick pace of development in favour of more secure, more stable platforms. Which is a good thing, I think. But nonetheless it results in a “slower” development pace.

The applications I’m complaining about are things like:
- SolidWorks (a CAD tool for designing parts and assemblies, used in manufacturing and mechanical engineering)
- SpectrumSoft Micro-Cap (a version of software similar to PSpice used by my school)
- AutoCAD (another CAD tool)

Luckily this is changing, but only for the large & most popular distributions:
- MathWorks MATLAB (runs on Linux and Solaris, etc.)
- Wolfram Mathematica (which has versions for Linux and MacOS X)
- FEKO (runs on Linux and Solaris among others)

Anyway, I still consider SolidWorks to be a rather big program not supported on Linux, which is a big issue for those working on Civil Engineering programs. There are most probably others which are very domain-specific that I don’t even know about.

There is a nice matrix comparing cross-platform capabilities of CAD software: http://en.wikipedia.org/wiki/Comparison_of_CAD_software

Oh, one final thought: perhaps that KDE Recommends: should be moved to a Suggests: instead, on account of its heavy dependencies, requiring mysql-server installed on desktop machines.. WTF!

Oh, and on another note, I re-installed Debian using the non-expert Auto Install and it installed Gnome rather flawlessly, much like installing Ubuntu, which was pretty nice. So kudos to those who have been working on the main installer; it seems as though the advanced ones really give you some rope to hang yourself with, though :-)

Oh, and k3ninho told me that there is an initiative from the Ubuntu community called “100 Paper Cuts” to help fix small bugs like those I was complaining about. I hope this leads to an improved user experience, and I’d really like to see some of those changes propagated both upstream to Debian and upstream to the KDE folk.

During my install of Kubuntu + KDE, I felt that plasma was crashing more than Windows Explorer — it felt like the days when I was running Windows ME, and the shell would crash, and the system would restart it. Repeatedly. It’s exactly what seemed to happen with plasma. I’m not sure if it was something I screwed up during configuration (presumably so), but KDE was far too complicated for me to try and debug. It might also have been a result of me running my system within a fairly constrained virtual machine environment – the system only gets 768MB of RAM and no access to the actual graphics processor unit (since it’s virtualized).

Recently there was a thread on the Google Summer of Code students’ list discussing gender dynamics in open source, but more broadly, interactions between those of different genders (mainly the discussion was simplified to be a discussion of sexes, which I think demonstrates the lack of understanding of the difference between gender and sex. But I suppose that’s a blog post for another day).

It was noted that many of the women on the list have blog addresses and other details that quickly self-identify the authors as female. There was discussion about whether this is a good thing or not, and the possible reasons behind it.

Here is what I wrote:

I think what you mention about yourself shows the world what you think about yourself, and what you consider yourself.

If first and foremost you associate your identity with being female (or male) or straight (or not)… then I guess that’s your prerogative.

But I, for one, am not /just/ an Asian male. I’m not just a Computer Science student. I’m not just a coder. I’m not just an Engineering student. I’m not just 20-years old. I’m not just a blogger. I’m not just an Open Source contributor. I’m not just an advocate of strange and often unpopular ideas.

I am a human being, with many dimensions. And I don’t try to simplify it by putting myself in a box and categorizing myself as anything.

I think that the key is just to understand everyone for who they are, and part of that is being somewhat ambiguous. As Leslie [Hawthorne] somewhat alluded to, it’s about managing people’s preconceptions about you.

I do not actively try to hide that I am male, or that I am Asian (you might guess that from my last name). There are all sorts of preconceptions people might have about things, and there are lots of -isms we should seek to avoid. (I’m Asian – maybe that means I’m a bad driver, and that I can’t pronounce Rs. I’m male – maybe I’m violent. I’m in Computer Science, presumably that means I play Dungeons & Dragons with my classmates on the weekends. I’m in Engineering, maybe that means I’m sexist.)

The reality is: none of these things should matter, nor should they define you.

Just be yourself. You show to the world what you consider relevant about yourself.

And for what it’s worth, I found out the other day that someone I respect and admire in the open source community is a teenager. Somewhere around 15 years old. It’s impressive, really. I look up to him, because he’s a really smart guy. But that wasn’t something he brought up right away; his nickname wasn’t “smartdude15″ or anything
like that. That’s the magic of open source, and the Internet — I judged him purely on his knowledge. And once I did find out, I thought to myself… Wow, would I have thought the same thing of him if I knew his age right away? Would I have even given him a chance, or would I just dismiss everything he said as something an immature teenager might say?

I think along with sexism there are tons of other issues to worry about, like racism (consider how difficult it is in some cultures, and even in Western culture, to be really accepted if you are gay, lesbian, transgender, bisexual, two-spirited, asexual, intersex…) In fact, being gay was considered a disease until relatively recently.

I’m glad for all the progress women have made in the past several decades. Not everyone has reached a point where they are accepted in mainstream society, and not everyone feels comfortable announcing certain details about themselves.

If *all* you are is a woman in a male-dominated world, then I feel sorry for you. I truly, truly do. Because none of the women I respect and admire are that. They are, first, talented Engineers, Scientists and Programmers, who are only incidentally female. Being female isn’t something that really identifies them any more than the colour of their skin, hair or eyes. No, no, they are talented, and that is, in the end, all I care about, and that is one reason I am grateful for Open Source — because you oftentimes don’t meet the people you are working with all the time in real life, so you cannot judge them on anything other than their ability.

Okay, so allow me to explain my personal development platform. I use Windows XP Professional as my primary operating system for various reasons, but I develop software for Linux (mostly because I like Linux, but also because I intend for a lot of my software to be used on my own servers, all of which can be considered Unix-like).

Because I prefer Windows for daily use, I’ve built various tools around the operating system and have become quite attached to them. I really like TortoiseSVN and Notepad++, for example. I’ve just gotten used to running Windows.

Anyway, I wanted to see if I could transition into using Linux for everything, and because I work in Debian and have servers running Debian, I installed Debian. I had some trouble at first, prompting me to change to Kubuntu, but later came back to Debian after realizing that the packages in unstable and testing that I’d come to love were under an entirely different process in Ubuntu, which is currently unknown to me.

So long Ubuntu, I’ve re-installed Debian and love it so far. One problem I’ve noticed is that Debian would not boot inside VMware Workstation under Windows, while Kubuntu was able to do so out-of-the-box. Instead, I got an error like this:

WARNING bootdevice may be renamed. Try root=/dev/hda2
Gave up waiting for root device. Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/sda2 does not exist. Dropping to a shell!

Looking into it further, it turned out I could edit the command line options to use a different device name–the debug output was right, I had to use /dev/hda2 instead of /dev/sda2–and things would work. The system would boot up normally, and everything was good. I guess it had to do with my drives actually being SATA (which I suppose are treated like SCSI drives); whereas VMware Workstation (in my setup) presents them as simple IDE drives.

But then I got to wondering how Ubuntu totally got around this problem, and it turns out that Ubuntu uses UUID identifiers to point to a hard drive. After searching on Google, it turns out that using UUIDs is convenient because it can handle cases where drives are removed; especially with external drives, or even re-arranging the drives inside your computer for whatever reason.

Using the command “blkid” from Debian (do this as root), you can get the UUID of all the partitions. You can either reboot into Linux using your normal dual-boot method (selecting the option from GRUB, etc), or you can edit the boot options from GRUB explictly to use /dev/hda2 temporarily. Once you boot into your Debian installation, you can edit /boot/grub/menu.lst to use drive UUIDs instead of /dev names.

Instead of:

# kopt=root=/dev/hda1 ro

or similar, you can change this to:

# kopt=root=UUID=<info from blkid for /dev/hda1> ro

Then you can use ‘update grub’ to update grub’s automagic menu.lst information, so that the new root is recognized. This way, however your disk is installed, the UUID will be detected and the appropriate drive mounted/booted. I haven’t tried this, but I suppose this would help if you have an operating system on a bootable USB drive or something, to help you select those partitions (though I don’t know if GRUB will even detect them on a boot. I’m hoping so since most BIOSes these days support booting from USB Mass Storage Media).

Good luck, I hope this helps somebody out there :-)

For my Google Summer of Code project, I have been working with PerlQt4 bindings, which requires that I have Qt4 installed. While this is technically possible under a Win32 environment. Lots of people in the free software community vehemently oppose Windows, but while it has its flaws, I think overall the hardware support is still much better than Linux. True, this is because of Microsoft’s shady business practices, and because many companies keep their driver source code closed. I’m still using Windows XP Professional and quite happy with it, stability-wise and feature-wise.

As an Engineer, many applications we use on a regular basis are simply not available on Linux. They’re simply not replaceable with the current state of open source software, though there is some great stuff out there. Nonetheless, we’re still far from a point where engineers in general can switch to Linux — the application support is as important to an operating system as the kernel. Linux would be nothing without GNU’s binutils, for example.

I tried to install Debian first, as this is an environment I’m very familiar with. I use Debian on my development server, and it has worked wonders there. But everything I do on that server is command-line stuff. When trying to install a desktop environment, I followed the KDE Configuration Wizard, which isn’t too bad, but it expects an Internet connection throughout the process. The problem was that I didn’t have enough Ethernet cables to have both the desktop computer and my laptop plugged in at the same time, even though I had a wireless router set up, which meant I had to unplug the computer while updating packages, etc. Some of the updates took quite a bit of time, which was inconvenient for everyone else.

I eventually got the system to install, and told tasksel to set up a desktop environment. It was installing stuff, I typed ‘apt-get install kde’ and assumed everything would Just Work. After installing a whole bunch of stuff (which included a local install of mysqld, on a desktop machine?! — turns out it was due to one of KDE’s recommended packages, it starts with an A, I forget which). Anyway, then the environment didn’t “just work” as I had expected. Upon booting up my system, it just dropped me to a command line prompt. Fine, I thought, I’ll just use startx. But that was broken too. So after another few hours of fiddling I just gave up altogether.

While trying Ubuntu (the last time I had done so was probably in version 7 or so), I downloaded a recent image of Kubuntu 9.04, the Ubuntu flavour using KDE as a default desktop environment. It’s surprising that there has been lots of progress in Ubuntu and Linux in general. I have found that driver support is much better than it used to be, as it now detects my network card – a Broadcom 43xx chip – and does everything it needs to do. For the most part, my operating system “Just Works.” Great. This looks like something I might be able to slowly transition toward, completely replacing Windows except inside WINE or a Virtual Machine container.

Has Debian and Ubuntu made lots of progress? Sure. I can definitely see that Ubuntu is geared a lot more to the average user, while Debian provides bleeding-edge features to the power user. Unfortunately, despite being involved in packaging Perl modules for Debian, I fall into the former category. I’d really just like my desktop system to just work. Oh, and dual monitor support out-of-the-box would be nice too — I hear the new KDE and Gnome support this.

One thing Windows handles rather well is changing hardware profiles – when my computer is connected to its docking station, a ton of peripherals are attached. When I undock, they’re gone. Windows handles this rather gracefully. In Kubuntu, I got lots of notification boxes repeatedly telling me that eth2 was disconnected, etc. This sort of thing is undecipherable for the average user, so I’d really just like for these operating systems to be more human-friendly before they are ready for prime time on the desktop.

One thing that makes Perl different from many other languages is that it has a rather small collection of core commands. There are only a few hundred commands in Perl itself, so the rest of its functionality comes from its rich collection of modules,  many of which are distributed via the Comprehensive Perl Archive Network (CPAN).

When CPAN first came on the scene, it preceded many modern package management systems, including Debian’s Advanced Packaging Tool (APT) and Ruby’s gem system, among others. As a consequence of its rich history, the CPAN Shell is relatively simplistic by today’s standards, yet still continues to get the job done quite well.

Unfortunately, there are two issues with CPAN:

1. Packages are distributed as source code which is built on individual machines when installing or upgrading packages.
   • Since packages must be re-built on every machine that installs it, the system is prone to breaking and wastes CPU time and other resources. (The CPAN Testers system is a great way module authors can try to mitigate this risk, though.)
   • Due to wide variation in packages, many packages cause problems with the host operating system in terms of where they install files, or expect them to be installed. This is because CPAN does not (and cannot) know every environment that packages will be installed on.
2. It does not integrate nicely with package managers
   • The standard CPAN Shell is not designed to remove modules, only install them. Removals need to be done manually, which is prone to human error such as forgetting to clean up certain files, or breaking other installs in the process.
   • It cannot possibly know the policies that govern the various Linux flavours or Unices. This means that packages might be installed where users do not expect, which violates the Principle of Least Surprise.
   • It is a separate ecosystem to maintain. When packages are updated via the normal means (eg, APT), packages installed via CPAN will be left alone (ie, not upgraded).

Here is the real problem: packages installed via CPAN will be left alone. This means that if new releases come out, your system will retain an old copy of packages, until you get into the CPAN Shell and upgrade it manually. If you’re administrating your own system, this isn’t a big problem — but it has significant implications for collections of production systems. If you are managing thousands of servers, then you will need to run the upgrade on each server, and hope that the build doesn’t break (thus requiring your, or somebody else’s, intervention).

One of the biggest reasons to select Debian is because of one of its primary design goal: to be a Universal Operating System. What this means is that the operating system should run on as many different platforms and architectures as possible, while providing the same rich environment to each of them to the greatest extent possible. So, whether I’m using Debian GNU/Linux x86 or Debian GNU/kFreeBSD x64, I have access to the same applications, including the same Perl packages. Debian has automated tools to build and test packages on every architecture we support.

The first thing I’m going to say is: if you are a Debian user, or a user of its derivatives, there is absolutely no need for you to create your own packages. None. Just don’t do it; it’s bad. Avoid it like the goto statement, mmkay?

If you come across a great CPAN package that you’d really like to see packaged for Debian, then contact the Debian Perl Packagers (pkg-perl) team, and let us know that you’d like a package. We currently maintain well over a thousand Perl packages for Debian, though we are by no means the only maintainers of Perl packages in Debian. You can do this easily by filing a Request For Package (RFP) bug using the command: reportbug wnpp.

On-screen prompting will walk you through the rest, and we’ll try to package the module as quickly as possible. When we’re done, you’ll receive a nice e-mail letting you know that your package has been created, thus closing the bug. A few days of waiting, but you will have a package in perfect working condition as soon as we can create it for you. Moreover, you’re helping the next person that seeks such a module, since it will already be available in Debian (and in due time it will propagate to its derivatives, like Ubuntu).

All 25,000+ Debian packages meet the rigorous requirements of Debian Policy. The majority of them meet the Debian Free Software Guidelines (DFSG), too; the ones which are not considered DFSG-free are placed in their own repository, separate from the rest of packages. A current work in progress is machine-parseable copyright control files, which will hopefully provide a way for administrators to quickly review licensing terms of all the software you install. This is especially important for small- and medium-sized businesses without their own intellectual property legal departments to review open source software, which is something that continues to drive many businesses away from using open source.

For the impatient, note this well: packages which are not maintained by Debian are not supported by Debian. This means that if you install something using a packaging tool (we’ll discuss these later) or via CPAN, then your package is necessarily your own responsibility. In the unlikely event that you totally break your system installing a custom package, it’s totally your fault, and it may mean you will have to restore an earlier backup or re-install your system completely. Be very careful if you decide to go this route. A few days waiting to ensure that your package will work on every platform you’re likely to encounter is worth the couple days of waiting for a package to be pushed through the normal channels.

The Debian Perl Packaging group offers its services freely to the public for the benefit of our users. It is much better to ask the volunteers (preferably politely) to get your package in Debian, so that it passes through the normal testing channels. You really should avoid making your own packages in a vacuum; the group is always open to new members, and it means your package will be reviewed (and hopefully uploaded into Debian) by our sponsors.

But the thing about all rules is that there are always exceptions. There are, in fact, some reasons when you might want to produce your own packages. I was discussing this with Hans Dieter Pearcey the other day, and he has written a great follow-up blog post about the primary differences between dh-make-perl and cpan2dist, two packaging tools with a similar purpose but very different design goals. Another article is to follow this one, where I will discuss the differences between the two.

When working on some code, I came across a random number generation algorithm (a pseudorandom number generator to be exact, since the numbers aren’t truly random but only designed to “look” like it to the casual observer) called ISAAC. The name stands for “Indirection, Shift, Accumulate, Add, and Count,” which is essentially what it does to a set of state variables, in that sequence.

The algorithm allows sequences of 32-bit numbers to be generated extremely quickly, since the operations it uses are relatively fast on modern processors; in fact, the author, Bob Jenkins, claims it only requires (on average) 18.75 machine processor cycles to generate each 32-bit value.

Even so, the algorithm is designed to be as uniformly distributed as possible.

I decided to test how fast this would be on a relatively modern multiprocessor system. The following is a benchmark on an amd64 machine with 2GB of memory. The benchmark script is available as part of the Math::Random::ISAAC package available via CPAN or its SVN Repository.

So as we can see, Perl’s built in rand() function is evidently really, really fast; quicker than every other algorithm studied here by a few orders of magnitude. What remains to be seen, then, is whether the quality of random numbers actually generated is any good. Ideally, most random number generators are designed to produce distributions of numbers that are uniformly distributed — that is, every number is equally likely to occur.

Uniform distributions often do not occur in real-life “random” samples – for example, things like height or marks in a class tend to follow more of a normal distribution (or “bell curve”) – the majority of the sample population is near the mean, and it drops off as you go to higher or lower figures. Nonetheless, these types of random numbers are great for uses in computer science — where you want each number to be equally likely, thus ensuring that your sequence is as unpredictable as possible.

In this case, all of the algorithms produce relatively uniform distributions, though the TT800 and Perl rand() core function produce a somewhat jagged distribution. Less interesting distributions are those for ISAAC, Math::Random and the Mersenne Twister.

While working on my Google Summer of Code project today, I came across a bug that pretty much halted my productivity for the day.

Early on in my project, I decided that working with Unicode is hard, among other things. Since I was restricted to using C, I had to find a way to easily manipulate Unicode stuff, and I came across GLib (I’m not even entirely sure how, I think I just remember other projects using it, and decided to look it up.)

Not only did it have Unicode handling stuff, it also provides a bunch of convenient things like a linked list implementation, memory allocation, etc. All in a way intended to be cross-platform compatible, since this is the thing that’s used to power Gtk.

I’m not entirely sure how it differs from Apache’s Portable Runtime (APR); maybe it’s even a Not Invented Here syndrome. In any case, I, not suffering from that particular affliction, decided to be lazy and re-use existing code.

For some reason, GLib’s g_slice_alloc() call was failing. For those of you that don’t know what this does, it is similar to malloc() in standard C – it allocates a chunk of memory and returns it to you, so that you can make use of dynamic memory allocation, rather than everything just being auto variables. In particular, it means you can be flexible and allocate as much or as little memory as you need.

So I spent the entire day trying to figure out why my program was segfaulting. Looking at the output of gdb (the GNU Debugger), the backtrace showed that it was crashing at the allocation statement. No way, I thought, that test is so well-tested, it must be a problem with the way I’m using it.

I changed the code to use malloc() instead of g_slice_alloc(), and the program started crashing right away, rather than after four or five executions with g_slice_alloc(). Not exactly useful for debugging.

I mentioned my frustration with C on the Debian Perl Packager Group channel, and a friend from the group, Ryan Niebur took a look at the code (accessible via a public repository). After a bit of tinkering, he determined that the problem was that I was using g_slice_alloc instead of g_slice_alloc0, which automatically zeroes memory before returning it.

It stopped the crashing and my program works as intended. I’m still left totally puzzled as to why this bug was happening, and I’m not sure if malloc isn’t supposed to be used with structs, or some other limitation like that.

But thanks the magic of open source and social coding/debugging, the expertise of many contribute to the success of a single project. It’s such a beautiful thing.

Update: There were a lot of questions and comments, mainly relating to the fact that malloc and friends return chunks of memory that may not have been zeroed.

Indeed, this was the first thing I considered, but the line it happened to be crashing on was a line that pretty much just did g_slice_alloc, rather than any of the statements after that.

For those that are curious, all of the code is visible in the public repository.

I do realize that the fixes that have been made are pretty much temporary and that they are probably just masking a bigger problem. However, I’m at a loss for the issue is. Hopefully the magic of open source will work for me again, and one of the many people who have commented will discover the issue.

The problem with secret questions is that they’re not very secretive. Back in the 1990s, it might have been reasonable to assume that information like “What was your first school?” might be secret. However, now, in the days of MySpace and Facebook, and especially Google, it’s fairly easy to find information about people.

A problem with passwords is that people forget them. On the other hand, a problem with other methods of authentication like public key cryptography is that they are not portable. Moreover, the problem with solutions like RSA’s SecurID token is that they can be lost.

Generally, the wisdom for truly secure systems is to combine these various approaches, in what is called Multi-Factor Authentication. Wikipedia describes these as:

• the ownership factors: Something the user has (e.g., wrist band, ID card, security token, software token, phone, or cell phone)
• the knowledge factors: Something the user knows (e.g., a password, pass phrase, or personal identification number (PIN))
• the inherence factors: Something the user is or does (e.g., fingerprint or retinal pattern, DNA sequence (there are assorted definitions of what is sufficient), signature or voice recognition, unique bio-electric signals, or another biometric identifier).

However, usually a password is sufficient because, if well chosen, and if the rest of the system is secure, they can provide a reasonable balance between everything. Yes, they do require memorizing, but on the other hand, you’ll never lose it unless you forget it. They are also weak because you can’t know if somebody else shoulder-surfs and catches your password, or has a keylogger installed on a public workstation.

Worse, if you are using an unencrypted wireless network connection such as the one we have at Western, then your data (including your password) travels over-the-air in plaintext. I’ve actually tried sniffing packets using Wireshark over the wireless network, and captured quite a bit of stuff, all without having to log into the system. For those in the know, there is also a secure network, but it’s significantly harder to set up – it uses WPA2 Enterprise – and though Windows XP and Vista both support it, the added cost of setting it up doesn’t seem worthwhile to most people. But all this is the subject of another article.

Because secret questions are often used for password recovery in the event that your password is lost or your account is compromised (by an attacker who missed the option to change your secret question), they are essentially a second password on your account.

Why are two passwords bad? They effectively double the chances that one of them will be cracked; if attackers find that your actual password is too difficult to crack, they might look at your secret question instead. Worse, because the question is considered “public” information (how are you supposed to remember the answer, without being given a question, after all), then attackers have a context for your password.

Imagine having this as your real password. “Hint: It’s the name of your first son.” People then don’t need to know you very well at all in order to figure out your password. Worse, most people will pass this sort of information without knowing it in conversation. Find out what someone’s secret question is? Steer the conversation there. “Got any kids? What’s his name?” etc.

This social engineering is particularly dangerous because while people know their passwords are precious, they are less likely to even remember their secret question and wouldn’t protect that information too much anyway.

Some better systems have opted to verifying e-mail addresses, but then the e-mail account becomes the weakest link. If a user’s e-mail account gets hacked, then the attacker thus has access to all their other accounts through the “Send the password to my e-mail” feature.

There are lots of solutions to this, but I think the lesson learned is that human factors play the largest and most often overlooked part of software design, especially web applications. For software security to improve, programmers and designers need to be a lot more careful – validate all input from forms/parameters, sanitize output that goes to users’ browsers to eliminate Cross Site Scripting (XSS) risks. It’s really just a reminder that a little bit of paranoia can go a long way to protecting the end-user.