Kevin's Weblog

I wish I’d brought my camera to work today. It is scary in Old Town, and not just the people camping out. Many of the businesses are boarding up their windows, as if they expect riots to break out.

I wanted to buy a Scaldis Noel glass from Lucky Baldwin’s, but apparently they hide all the breakables from the ravaging hoards, so I have to come back next week. I do wish they would put alcohol percentages on their menu, though. 12% !!! I might as well just go home for the afternoon, except that I probably shouldn’t drive either.

The death toll is now over 100,000 in Asia, and over a million people displaced from their homes. This could still rise substantially from resulting disease outbreaks. The most convenient way for many to donate money toward helping with this situation is though the Amazon Honor System, although they will only process payments up to $50. [Update: The limit might be higher than that, as the current average contribution appears to be about $60, and that page does talk about tax receipts for contributions over $250. However, I did get told that they couldn’t process my donation and the FAQ page says $50 is the limit.] If you want to contribute more than that, you need to go directly to one of the aid organizations, like:

• International Red Cross
• American Red Cross
• Doctors Without Borders

It can be difficult to decide what is the right amount of money to contribute in these situations and it is, of course, a very personal choice. For myself, the method I used was this: over the last month, I got some unexpected or unbudgetted money in the form of bonuses and stock option exercises. I took the post-tax amount from these, calculated 10% of the total, and rounded a bit, then donated that amount. For those fortunate enough to have gotten some kind of year-end bonus, I think this is a pretty reasonable sort of method.

Why is it that I totally hate project planning at work, but for a dinner party, I’m willing to write page upon page of inventories, tasks lists, and timelines at a level of detail that any sane person would have fired up MS Project to handle? I think this reflects sometime about the relative maturity of the two disciplines. Either that or it reflects that I’m a freak.

On the plane ride back to Maryland yesterday, I read The One Minute Manager Meets the Monkey. It’s a pretty fast 130 pages, but it definitely seems to fall into the category of management books that have a single insight that they simply repeat over and over until they fill enough pages to justify calling it a book.

In this case, the general insight is that you will be happier if you don’t take on tasks that aren’t (or shouldn’t be) your responsibility. If you are a manager, your reports will be happier too if you don’t take on tasks (monkeys) that they ought to be doing instead. A few techniques for getting out of the habit of picking up monkeys, and that’s basically the book.

Anyways, I’m not complaining too much because I needed something to read on the plane, and this book is a lot more compact than Quicksilver, which had to be relegated to my carry on.

Joel has an interesting article today on pricing software. I don’t think that there’s anything shockingly original there, but it does bring a lot of stuff together on one page. I would have liked to see some discussion on open-source / free software added (beyond the comments on the reality of demand curves), but I understand that Joel isn’t really in that business.

The local paper reported a couple days ago that “Social Security [is] a Gordian Knot”. I’m wondering if they actually put any thought into that metaphor, or if they just wanted to say “a big mess” and thought it sounded cool. (I didn’t actually have opportunity to read the article.)

The paper does have a conservative slant, so it’s possible that they really did mean it, as the popular opinion in that world does seem to be “in order to save Social Security, we have to destroy it.” Of course, the metaphor is completely false, as there are obvious and simple ways to unravel the problem (which isn’t actually anywhere near as bad as some would have us believe). Either of a small increase in the cap on Social Security deductions or a small increase in the age where benefits begin being paid out would be sufficient to get us over this hump. (Or a reduction in benefits, but that’s even more dead-in-the-water than either of those.) But, I suppose that that doesn’t make for a good headline.

It looks like the issue of inline advertising in RSS feeds is getting a lot of attention lately, with posts from Jeremy and John Battelle. I’m sort of surprised that Jeremy seems so hesitant about the concept. Personally, I’m really excited at the prospect of us getting into this area. But then, I work for the money-making part of the company so I’m used to viewing the web in a more commercial way than most. I also find the technical challenges of the problem really interesting.

There are some things I worry about, though. The main one is how various aggregators will react to this. I can imagine that some aggregators without another revenue model might want to put contextual advertising of their own beside feed content. The risk here is that the total quantity of advertising does go over some line and people rebel. Another thing that might happen is that aggregators might try to block ad content in feeds. Particularly if this is combined with them adding their own ads, I could see this leading to some nasty legal fights.

The crashes are back. They started slowly last week and have worked up to 4 or 5 a day this week. I guess I was too quick to blame Sailing Clicker, although I suspect that it was tickling something in the kernel. Briefly this morning, I thought that somehow something in fetchmail was to blame. My fetches had been failing since last Tuesday when my password got expired. After I corrected that, I was crash-free for the rest of the day, but after I got home I got several more.

At this point I’ve pretty much concluded that there is just something wrong with the image they set up for the pilot program at work, and a reinstall from scratch is the only way to save my sanity. I do have some evidence that they tinkered with things they probably shouldn’t have, as various programs I’ve attempted to compile have failed because they couldn’t find things like libmalloc. So, this evening I did a full backup. Tomorrow, I’ll start in on the install.

I’m not much of a sports fan, but I haven’t been able to avoid coverage of the Ron Artest controversy. I just had to wonder, is there any other profession where you could, after any provocation, dive into a crowd and start a fight by hitting one of your customers, and not expect to be fired? I guess I just don’t have much sympathy for these guys claiming that their suspensions are too harsh.

(I guess perhaps movie stars might also enjoy this level of privilege.)

I use two Firefox extensions: the Bloglines Notifier, and the Bookmarks Synchronizer. They are both cool and very useful and I couldn’t live without them. Now, with the 1.0 release, the Bookmarks Synchronizer became incompatible. However, I did find the new version after a couple days. However, when I installed it, the Bloglines extension stopped working.

Fortunately, when I uninstalled both, then reinstalled them, they both returned to functionality. However, this does seem to indicate a bug, and I’d have to guess that it’s a bug in Firefox. (In fact, even if there is a bug in one of the extensions, the fact that installing one extension can cause another to completely stop working seems to suggest a design bug in the extension framework.) At any rate, I filed a Bugzilla bug, but they are insisting that I should sent it to the extension authors instead. However, it seems that neither one wants to know about bugs since there is nothing on their webpages to let me know their email addresses. (The Bookmarks author invites me to send him email via some forums website which requires registration to do it, but WTF?)

Come on people. If you’re going to release software, please make it easy for people to contact you.

Tomorrow we’re heading off for a long weekend trip up the coast. We going to a tiny little town that we are told has nothing to do, where we intend to do just that, perhaps with a jaunt over to Paso Robles to acquire a couple bottles of wine to accompany us in our nothing. It’s going to be great.

I have a new reason to dislike the Electoral College. Not because it overrepresents rural areas. Not because the winner-take-all allocations distort the popular vote. Today it is because of the realization of how much it contributes to the appearance of deep division in our country because of the binary red state-blue state maps that come out of it. Well, the next time someone tells you that some state is red or blue, you can tell them that they’re all lavender

Even more interesting in some ways is this county-by-county map. What I love about this is that it shows that there are more bastions of solid Democratic support in the south than on the west coast or the northeast.

Update: one more variation, again vectored by jwz, this time a purple cartogram.

This worries me (from Will Saletan):

Kerry led 51-49 in the Ohio exit poll this afternoon. But he also led 51-49 in the Florida exit poll, and we've seen what happened there. Nationwide, the exit polls had Kerry up 51-48. But with 80 million votes counted already, it's Bush who has a 51-48 lead. So at this point, the exit polls are at best meaningless. Or worse, if you're a Democrat, the six-point gap between what the exit polls predicted for Kerry nationally and what the returns show so far means that in Ohio, a two-point lead for Kerry in the exit poll foreshadows a Bush win by as many as four points.

I worry about whether provisional ballots are actually going to be counted, not just in Ohio. In the first nationwide use of them, it seems critical that we count them all. Many people expect them to swing strongly Democratic, and even if it’s not enough to save Kerry (and it probably isn’t), the data is important.

I’m also pretty upset with Kerry’s logic of it being better for the election system to concede. Why is it not better to actually count all the votes? Doesn’t this just tell all the people who might be first-time voters, or move frequently, or whatever reason for casting a provisional, that their vote doesn’t really matter? Do we not want them to vote next time?

I must say, OBL’s new tape is a mindfuck of the highest caliber. Total world series of geopolitical poker. I have a feeling you’ll see people second (and third and fourth) guessing this for the next three days and then some. As in poker, it’s probably best to just play the cards you’ve got and not try to figure out what type of psychology your opponent is using. However, I would be surprised if there isn’t a noticable jump, probably in Bush’s direction, in the polls over the next day or two.

We’ve got two props on the ballot involving taxes. 63 hits $1M+ earners with an extra 1% to pay for mental health care, and 67 adds additional fees on telephone services to pay for EMS services.

63 is designed to be popular (or populist) with it’s sock-the-ultrarich approach. The won’t-hurt-me argument almost won me over, but on further thought I come down against this. The first point is that CA income taxes are already extremely progressive and high. The rates ramp up quite quickly and those with more than $50K or so of taxable income see a 9.3% marginal rate. When I consider this and ask myself if I think that the tax structure ought to be more or less progressive than it is, I find that my answer is less. Beyond this argument, there is also the fact that many rich people already arrange to “live” in states with lower taxes. I suspect that the opponents are correct that this sort of vindictive taxation will just make that phenomenon worse.

I have some reservations about 67, primarily because of the slightly dishonest way that it piggybacks on the 911 service fee, and because I’m generally opposed to the way that the state has choosen to tightly couple specific revenue sources to specific expenses. However, I do think that it is critically important to keep emergency rooms open. While it is true that a substantial amount of this money would go to private hospitals, the fact is that they are required by law to provide emergency services to all individuals even if it is unprofitable. It’s completely reasonable in my might to provide subsidies to encourage companies to actually do this, rather than closing their emergency rooms completely.

Holy crap! The Economist just endorsed Kerry. Granted, this is probably the right way for them to go given their biases and values (responsible fiscal and international policies, and lack of government intervention on social issues), but it comes as rather a surprise given how down Lexington has been on Kerry and Edwards.

I’ve been meaning to comment on the propositions on the CA ballot for a while, but haven’t had a chance to really dive into them until this week. Hopefully, this will be the first in a series of posts covering most of them.

The two election related props seem like pretty straightforward choices to me. 62 is a clear NO. Fewer choices on the general election is bad, not good. In addition, anything that allows crossover voting without other election reforms opens the door to all kinds of gaming of the system.

Prop 60 doesn’t actually change anything about the current system. However, because these two propositions are in conflict with one another, it is important to vote YES on this one in the case that a tie-breaker is needed if both pass.

(Amusing note: The “Independent Voters Guide” that I got in the mail today urges a NO vote on 62 because if it passes there could be no Democrat in the general election for some districts. Uh... aren’t we supposedly independents? And isn’t this California, where the opposite seems more likely?)

There’s been a couple interesting articles at The Ninth Bit in the last week. Megan explains Why She is a Libertarian which tends to agree with my libertarian leanings. (Although I have to say I would never label myself a capital-L Libertarian.) I’m glad to see that I’m not the only person sympathetic to most of the libertarian platform, but uncomfortable with the gun-whacko-ness.

I found the link to this article, Morlocks vs. Libertarians interesting. Somehow I’d never considered that someone might be seriously trying to pull a Nader from the other side. I have noticed, though, that in polls that include Badnarik he tends to do at least as well as Nader.

I do have to say, though, that this “taxation is slavery” stuff is crap. You can renounce your US citizenship and move to a country without income and property taxes. As long as you are free to leave, you are not a slave, but a willing participant in a social contract.

Can someone explain to me exactly what it is that credit card companies mean by “close an account”? Because it clearly isn’t what I mean by that phrase. I’ve closed three or four credit card accounts in my life and they all still appear as active on my credit report. Furthermore, in all cases, charges have continued to show up on the accounts after they were supposedly closed. Usually this is things like regular bill payments, but in two cases they were fraudulent charges.

Yesterday I came home and found a letter claiming that my Sears card, which I haven’t used in 7 years and which I canceled nearly two years ago, had a $55 balance that had been sent to collections for non-payment. The number of things wrong with this is just amazing. First, the account is closed. Second, this is the first I’ve heard of it, despite the charge in question getting made back in May. But it gets worse. They claim that in April, a $9000 payment was made to the account to cover the balance at that time! At this point, I’m starting to suspect that their database has gotten corrupted somehow. Fun.

The phone conversation started out really interesting. The letter didn’t actually include the account number. So it took a couple minutes of going around in circles before I could get them to even start dealing with the problem. “I have a letter from you claiming I owe money on an account that’s been closed for 2 years.” “What’s the account number?” “I don’t know, I closed the account 2 years ago.” “Well, I need to know the account number to help you.” “So, what am I supposed to do about this letter?” “What’s the account number?” ...

Look Apple, it really doesn’t make me feel any better when my computer crashes that I get a really pretty translucent screen of death, instead of an ugly blue one. In fact, it pisses me off that someone spent time making that happen instead of fixing the fucking problem that caused the computer to crash in the first place.

While I’m on this rant, I’d like to point out a big problem in the whole “The Web is the Internet is the Computer” thing, which is that my local desktop apps are pretty good about periodically saving my work without me having to do anything. However, when I’ve spent the last hour typing into a little box on a web page (which is a crappy way to do text editting to begin with) and then the network goes down or someone trips over a power cord or one computer or another crashes, I lose all my work.

As if to prove my point, the computer just crashed again before I could upload this to the server. However, I didn’t have to retype a thing.

Update: After 5 crashes yesterday, I gave up for the day. This morning on a hunch I uninstalled Sailing Clicker and so far today I haven’t had any problems. I’m somewhat disappointed as I think it’s a neat piece of software and I was looking forward to playing with more of its features, but at this point I haven’t even given a presentation since I installed it and clearly I can’t afford this level of instability.

Charitable organizations and I seem to be having a hard time of it this year. Yesterday, for the fourth time this year, I declined to make a contributation that I normally would have. All I really ask is that they not annoy me, but that seems to be an unreasonable request.

1. I did not renew my membership to MOCA this year. This was mostly because I hardly ever go. I hardly ever go because whenever I consider it, I go to their website to see what it currently on display and I rediscover how completely unusable their it is. I’m not just talking about bad layout or that sort of thing. Rather, they put all their content in this wierd custom scrolling box which has the unfortunate property of scrolling diagonally rather than vertically, so you can’t read past the first page.
2. Consequently, when I am in a museum mood I usually end up going to LACMA. Except that I decided not to renew that membership this year, mostly because they (or at least their security guards) are extremely un-family friendly and I end up getting in an argument with someone nearly every time I go. They have also been misleading in their marketing of various special exhibits.
3. I didn’t renew my KPCC membership because I was disgusted about listening to commercial advertisements on my “non-commercial”, “listener-supported” radio station, as well as the wierd product placement that seems to be sneaking into some segments.
4. Then, last night I gave up on donating to the Tour of Hope. A friend is riding and I wanted to sponsor him, but the online form had some broken Javascript validation that kept popping up windows claiming that my caps lock was on and that I should turn it off. After a couple of those, I gave up on the process.

I think that people think that these things (bad web user experience or quality of guards) don’t really matter. Sometimes I wonder if I’m just being petty. But the thing is, these people are asking for my money and it seems like in return I should expect, at a minimum, that they don’t piss me off. It that really unreasonable?

There’s something odd about the news coverage of Clusty. The New York Times reports that the output is based on results from Overture. However, as far as I can tell, only the clearly marked sponsored results come from us. I’m not sure where the search results come from, but it’s somewhere else.

As usual with Philip, I have mixed reactions to his latest article. I agree with the premise that the typical CS undergrad education in the US doesn’t really prepare people to be successful software engineers. Certainly, I share the opinion that more group projects, and more large, open-ended, and real-world-like projects are needed. I also think that a punch-in-punch-out mentality has to be avoided. (Although unlike Philip, I don’t find that inconsistent with a FortyHourWeek; if you aren’t dedicated and engaged, working longer won’t make you that way.)

I found that this excerpt echoed a lot of my current feelings:

Second, we want our graduates to have satisfying careers where they have autonomy, responsibility, and job security. Consider the typical computer programmer who takes specifications for a subsystem from a manager, unthinkingly produces a program to meet the spec, and hands the resulting code back for testing, integration, and evaluation by other departments. This is really a factory job where the workers' hands don't get dirty.

What's wrong with that? Sociologists have tracked a substantial loss in worker satisfaction as craft jobs were replaced by factory jobs. A craftsman builds a complete product and delivers it directly into the hands of a customer, seeing and hearing the satisfaction that the new product delivered. A factory worker, by contrast, builds a small component of a product and is typically several distribution layers removed from the customer.

A craftsman is responsible for whether and when a product goes out the door to the customer. In a classical factory a totally separate part of the bureaucracy, the quality control department, decides whether or not a product should be shipped. A craftsman listens to customer needs and adapts the resulting product accordingly. Factory workers never talk to customers; a separate marketing department assesses customer needs and determines changes in product design. Not only is a job in a code factory not very satisfying it is neither very secure.

What don’t I agree with? Well, to start with, there is the continuing fear of a brown man. Yes, lots of Indians and Chinese are getting technology jobs. And, good for them! However, I’ve supervised outsourced development groups and I’ve interviewed many developers, and it isn’t at all obvious to me that the Indian or Chinese schools are doing any better than the US ones. The Asians I’ve interviewed in general seem to have retained their ‘book learning’ better than the Americans, but they don’t do substantially better or worse at puzzles or problem solving on average.

I also don’t approve of the idea of removing the traditional liberal arts from the curriculum of aspiring software engineers. We can improve education within the discipline without removing other disciplines. Furthermore, a successful engineer of the sort that Philip would like to see is going to need to be able to communicate effectively, in speech and writing; to read critically and assimilate information from diverse sources; to understand political and macroeconomic trends, as well as the microeconomic forces at play within a business.

Finally, I would add a few suggestions, which might be implicit in Philip’s vision, but which could also be implemented within the current environment. First, students must be required to use version control on any project which involved multiple people, or which spans more than one week. Second, from day one, students should be expected to hand in test suites with every assignment, instead of relying on the test suites provided by TAs.

I was fairly impressed with Kerry’s improvement in delivery in the first debate, but in the end, I’m not sure that it’s enough. His position on Iraq is a tough one and it’s hard to sell. I mean, “how do you ask a man to be the last to die for a mistake?” His Pottery Barn analogy was just a little bizarre, and I think can only be explained by some staffer saying something like: you have to capture the Soccer Moms (oops... according to NPR it’s now Security Moms) vote — toss in something about Pottery Barn or something. Beyond its bizarreness, I will be amazed if it doesn’t get turned around on him: “Kerry says the government broke it, and now he wants your son to buy it.”

Other than that, Bush made a lot more gaffes, although Kerry let him get away with virtually all of them. Some examples:

In response to a question about the importance of Bin Laden vs. Iraq, he slipped and said, “Of course I know that Saddam Hussein is the most important... I mean that Osama bin Laden is the most important...”

Bush would say to troops, “I will bring you home as soon as the mission is done,” apparently forgetting about the mission accomplished stunt. (Kerry made a quick reference to this, but only an hour later. He should have jumped on it immediately.)

Bush looks like quite a bit of a hypocrite going on about the International Criminal Court after the events at Abu Ghraib.

Bush several times insisted on a 1-minute follow-on round and then appeared to have nothing planned to say.

One point of criticism for both candidates is that apparently they both feel that Iran is more important than what’s happening in Darfur, since they both chose to spend the first 30 seconds talking about that country instead of answering the question that was asked.

Sadly, my feeling here, though, is that although Kerry really topped Bush in terms of actually addressing issues that were put forward, unless his campaign can really work the spin engine, the conventional wisdom is going to be that Bush won it because he looked at the camera more and because he appears to have stronger convictions (i.e. is incapable of admitting to any mistake).

when you find yourself wondering, does Ciara really mean what she says in her new single: "Just because you drive a Benz, I’m not going home with you", or did she really intend to say "I’m not going home with you just because you drive a Benz"? Should hip-hop artists be required to understand the difference between dependent and independent clauses?

Today I went and retrieved my books that have been sitting in my old Caltech office for 4+ years. It was a little wierd how little the office has changed. Comics I stuck on the wall are still there, and the desks look more-or-less like they did when I was there. I glanced at the titles of a couple of the papers scattered about and it was all the sort of stuff I would have read. My books were up on the top shelf, but the shelves of the current occupants contained a fairly indistinguishable set of books.

The big thing I forgot when I was figuring how many boxes I would need was my notebooks. I’m not sure what I’m going to do with them, but I now also have several hundred pages of notes from 2.5 years of classes, and research. I did decide to leave the e-prints I had collected in the file cabinet.

I did a quick triage of the books and figured out which ones I’m willing to part with right away. They’re now on my forsale page, on the miniscule chance that anyone reading this is interested in graduate level physics and math books.

A new feature — my Perl Hall of Shame, featuring occasional snippets of perl code that someone actually wrote. Today, a little pattern I’ll call “useless use of $_”:

for ( 0..9 ) {
  my $i = $_;
  for ( 0..9 ) {
    my $j = $_;
    ...
  }
}

I think that perhaps the author of this was trying to avoid polluting the enclosing lexical scope with his loop variables, but wasn’t aware that exactly the same thing can be achieved without resorting to punctuation variables:

for my $i ( 0..9 ) {
  for my $j ( 0..9 ) {
    ...
  }
}

It’s been one of those rollercoaster weeks at work. I really need to figure out how to get my zen on. It’s suddenly become clear to me that the turf wars being fought are much bigger than I realized and that probably no amount of reasoned argument from us down in the trenches will have any impact. Possibly coding like mad will, but I’m not sure; it might just lead to coding like mad in a different direction later. At any rate, it may well be best for my sanity if I just duck my head down for a couple weeks, ignore it all, and get shit done.

This week, I’ve gotten two emails out of the blue from head hunters, both just completely mis-targetted. So, beyond me learning that I should perhaps remove one or two keywords from my resume, I have a couple observations / suggestions for recruiters:

• Where it says “I am not currently looking for work”; I mean it.
• I am not really impressed by your assertion that the job is “very secure” because you’ve had someone working there for 2.5 years.
• 9K concurrent users also does not impress me.
• I’m probably not interested in working on stuff I did in my first year out of college and haven’t touched since.

I have to say that in the middle of a week full of various people suggesting your code is responsible for the company losing scary amounts of money, breathing down your neck for not having fixed the bugs in other people’s code yet, and accusing you of being unprofessional (read “mean”); it is really nice to have someone pull you aside and say “Hey, did you fix bug X in the last launch? Thanks, that was great.”

I picked up Paul Krugman’s The Great Unraveling today. Reading through the introduction, I just about stopped dead at the admonition:

Don’t assume that policy proposals make sense in terms of their stated goals.

I was taken aback by the fact that this was nearly identical to a point that Schneier hammers on in Beyond Fear. For the most part, Schneier is talking about security policy and Krugman is talking about economic policy, although they both make the point that proposed policies in their respective areas may really have more to do with broader goals.

I suppose that in some sense this convergence of ideas isn’t actually too shocking, since both authors actually have fairly similar agendas: to argue that the current administration, aided by the events of September 11th, is making very questionable decisions in order to forward questionable goals, and that this is all fairly obvious if you just think about it clearly.

At Barnes and Noble, they have a display of “banned books”. My first reaction was to note how many of the books on the table I’ve read. My second reaction was to realize just how much more interesting I would have found the display if they had some additional information on why each book was banned, and by whom.

If I ever do get around to taking some classes “for fun”, maybe I’ll have to remember to look for a lit. class on banned books. I imagine there are some interesting stories surrounding the stories.

I just read John Battelle note about Geico’s lawsuits and thought about the supermarket. Most supermarkets hand you a couple of coupons along with your receipt. If you look at them, you quickly realize that most of the time they are coupons for a competing product to one of the items you just purchased. I wonder how this works. Do supermarket advertisers “bid”, as it were, on specific competing products or just on categories? Is this a relevant parallel situation to Google and Yahoo?

(This seems like a good time to point out that I work for Overture/Yahoo, but that nothing I say here in any way represents the opinions of my employers.)

I pretty much keep NPR tuned in the car most of the time, although this week has been tough. In principle I’m actually interested to here what’s going on at the RNC, in a know-your-enemy sort of way. However, I’ve generally found that I actually can’t listen for more than 30 seconds.

While I didn’t catch the whole speech, I got highlights of Arnold this morning and it was completely different. I mean, analytically, he was vastly oversimplifying and ignoring the inconvenient, but it was still excellent rhetoric. This man is dangerous.

I think the whole model year thing has gotten completely out of control. I want to buy a new bicycle, and REI is having a sale, including bikes. However, when I just called to find out what they have and what’s on sale, I found out that they have almost nothing in inventory at the moment, because they are waiting for the new ’05s to arrive.

So, maybe there’s something wrong with me, but I really don’t see why bicycles need model years. I understand that manufacturers may periodically want to change the components on a particular model, but I can’t reasonably figure out why this should need to be synchronized to the calendar.

Okay, it totally sucks that Benchmark only looks at CPU time. Honestly, how often is CPU the bottleneck in a real systems? Granted, it’s pretty simple to write an alternative benchmarking tool for wallclock time, but that’s just all the more reason that it ought to be available as an option in the standard package.

One of our partners, who shall remain nameless, recently reported that the error rate on their queries to us had gone up by a factor of about 10. After a day or two of our tier 2 staff looking into it without identifying the problem, I was pulled in last night. The first thing I asked was, “What do they mean when they say ‘error’?”.

This morning I reviewed what had been done so far with our tier 2 guys, concluded that they had done everything I would have thought of to do and that the problem didn’t appear to be on our end. I then emailed the partner to say that we suspected that the cause was elsewhere, but that we would be surer if they could give us a histogram of the types of errors they were seeing. About an hour later, they came back and told us that it turned out they had accidentally launched some code they shouldn’t have and that the errors were all 404s as they sporadically queried the wrong URL.

Sigh...

There was discussion about Walmart on Marketplace this evening. Apparently someone outside the alternative press finally realized that when companies pay below a living wage and expect their employees to make up the difference by applying for food stamps and the like, in essence the taxpayers are subsidizing the operation of that company. This isn’t exactly rocket science, but I guess it’s news to some.

However, there was a counterpoint from someone arguing that Walmart employs mostly retirees (hmm... employeed retirees, how’s that work?), college students, and the variously disabled, so they’re actually reducing the public burden because otherwise those people would be completely dependent on public assistence. Now, maybe I haven’t seen a representative sample, but that doesn’t match any Walmart I’ve been to, where the employees pretty uniformly appear to be typical working-age, working-class, fully-abled individuals.

In other recent news, I saw a couple newpaper headlines insisting that there was no post-convention bounce for Kerry. However, the Electoral Vote Predictor strongly disagrees. Liberal media indeed...

I think it would be interesting to see a similar graph, but distorted so that size reflected population or electoral votes. Already this graphic is nice in that it reflects a shift from the vast sea of red that we’ve gotten familiar with in recent years, but shrinking some of those wide-open western spaces down to size would really paint a different picture.

Well, looking back, it’s been a pretty good week. Really busy, but interesting and worthwhile. A couple sessions disappointed me, and there were a few things the conference organizers could have done better, but on the whole it was quite valuable.

My big synchonicity of the week revolved around Damian Conway, Paul Graham, and Brian Ingerson (IO::All). I had actually been rereading some of Paul Graham’s essays before the conference, including Programming Bottom-Up, a concept that he touched on briefly in his Tuesday night talk. The basic idea is to extend the language to match your problem domain, thus making it easier and more natural to express your programs. Ingy has created a few examples of this in IO::All and Spiffy. Damian talked about this and more in his Sufficiently Advanced Technologies talk.

At a minimum, this got me to think about some of my basic utility classes and how I could modify them to be more natural to use. But, I’m also thinking about new development and how to keep things cleaner. I need to give some thought, though, to understanding how to ensure that this technology is used for good. If you have a small group of highly talented developers, it is clear that using this power is the way to go. What is less clear is how to scale it to larger development groups. How do you educate people if you change the language itself? And, how do you ensure that this sort of deep magic is used carefully and with restraint?

My other big conclusion was that we can definitely contribute more to this conference. There are interesting problems we are addressing which I didn’t see anyone talking about. To a certain extent, we will be limited by confidentiality, but there should still be general stuff we can talk about that isn’t specific to our application or so critical to maintaining competitive advantage. So, I will probably submit a talk next year, working title “How to serve 1,000,000,000 requests a day with mod-perl”. Keep your eyes open for it.

With one child in college and three to go, the Kent started thinking about the economics of software. Concluded that open source is nonsense. Not saying he won’t do it, or that you shouldn’t, but just that you should be aware that it’s nonsense when you do it.

A goal of society is to apply scarce resources where they do the most good. Money is a tool to do this. Money is information about what is valuable in society (mostly, not perfect). How does the OS movement figure out what is important to do if not with money? Prediction: open source will result is really good software with minimal impact.

Ex., rough estimate of value created by JUnit is $1 billion / year. Kent Beck’s profit: $0. Geek cred is nice, but doesn’t pay for college (or food, or rent, ...).

Maslow’s hierarchy of needs: survival, love, esteem, self-actualization. Must fulfill lower levels before you can proceed to higher levels. But, open source turns this completely on its head. Do what you want, maybe then get recognition in community, maybe then form close relationships, only very few manage to fulfill survival needs through OSS.

Pricing in traditional economic: price set by a seller is a statement of worth. Price paid by a buyer is a statement of perceived value. We (OS developers) think we are making items of value, but to most of the world a price of $0 signals otherwise.

Situations that don’t make sense are warnings of impending change. In what direction will change happen? Hard to say, but if we are aware that change is coming, maybe we can steer it in a direction that is good for the creators of value. But, if you don’t pay attention, MS will capture all the value instead.

Accountability: software development doesn’t have a good history of accountability. Increase in test-driven development is a step towards accountability. OSS tends only to have peer accountability. The problem with your peers is that they tend to let you off the hook. “Oops... it was 3AM and I made a mistake”.

Why does OS seem to work? The first world is a land of plenty. If you have a technical background, you can’t fall that far. At least in the short term, people are willing to sacrifice basic needs for higher needs (doesn’t work so well in the long term, though. Do need sleep, housing, food). Programmers are willing to write software instead of being involved in other hobbies or local community.

How could it work in the future? “Whole people working whole jobs”. Sponsorship (why not USPS JUnit instead of cycling). Patronage (shortage of interested wealthy people, audience member recommends Public Software Fund). Pay-per-use (direct feedback for developers). Licensing. Complementary products.

Preable: big “thank you” to Larry Wall for making this all possible. If everyone in this room got their company to donate $500 to The Perl Foundation, it would be more than enough to keep Larry working on Perl for the whole year.

Desire from someone: implicit $self-> for method calls inside a class module. Sub::CallAsMethod does this. Very simple. Very magical. Probably evil.

This talk is about modules that are indistiguishable from magic.

hmm... blah blah blah... Harry Potter jokes without end...

Modules which do magic without any user interaction aren’t new. Most famously, use strict. More recently, IO::All. But, maybe not magical enough...

use IO::All:Pulp::Fiction
my @horrors = <~/foo.txt>

Lots of stuff in the Perl6:: namespace.

How can mere mortals write S.A.T.? Find the clunky bits of your code and do something about it! IO::Prompt was pretty cool, but originally needed too many flags. Changed to autochomp by default and autodetect boolean context.

Why can’t comments be more useful and interactive? Progress module presented last year weren’t simple enough to use. Solution, smart comments:

use Smart::Comments
while ( ... ) {     ### Preparing--->done
  do_work
}

also provides checks and assertions.

Wrapped up with Lingua::EN::Autoinflect, which is like Lingua::EN::Inflect, but with no interface, just works.

Takeaway: look at the interface of your modules. What could you remove? What could you just infer? Could you have better defaults? Could you use objects with overloaded operators to acheive things with more natural usage?

It’s pretty hard to do this justice, but check out the David Rumsey Map Collection. Amazing historical maps with the ability to compare, overlap, evolve, and knit maps.

Big sites run on Perl: Yahoo / Overture, Amazon / IMDB, TicketMaster / CitySearch. Billions of requests a day served with Perl. How do we do it?

Scaling at this level requires more than just hardware. If your application is slow, you can’t just throw hardware at it. This is about high-level design patterns for applications.

Caching

It’s nearly always worthwhile to cache some data. Caching full pages is best if you can do it. Pregenerate, or generate-on-the-fly and use mod_proxy cache. Usually can’t do this because you have at least some personalized data on every page. Partial page caching caches different page sections with different expirations. Mason provides this. Other modules: Cache::FastMmap (key/value storage in local shared memory), Cache::Memcached (distributed key/value storage)

(Aside: tied BerkeleyDB bad — object interface about 3x faster)

Job Queuing

Bursty traffic is a real pain. If traffic load exceeds steady state capacity, you need to queue requests to be handled in turn. Decouples users from backend load. Typical “search in progress” approach is okay, but doesn’t scale unless you limit the number of worker threads / processes. Don’t want to fork 1000 processes if you have 1000 user waiting. Spread::Queue is one possibility.

Questions...

How do you deal with session data? General rule: keep as little session data as you possibly can.

Memcached vs. MySQL performance? About the same for small numbers of clients. memcached ought to be better for large numbers (at least on BSD or Linux 2.6)

Is Spread ready for production? Seems to be.

Damn... been coding for 5 minutes and I’m wishing I could use IO::All.

Well, I’m figuring I picked the right talk for this time period, since Damian Conway is in the back row as well as a couple other luminaries. If it’s worth his time, it’s probably going to be interesting. (Correction: everyone is here)

Goal: make all i/o as easy as writting to STDOUT. E.g.

"Hello World\n" > io("greetings.txt");

Don’t worry, there are method versions of everything, but overloaded operators seem to make it clearer.

IO::All exports the io() method which is just a factory for IO::All objects. IO::All objects can be files, directories, sockets, pipes, dbms, web pages, and more.

files can slurp(), >>, or scalar ref to get contents.

next() iterates through directories, returning more IO::All objects.

Stuff that just works:

io(‘—’) > io(‘-’);
$tmpio = io(‘?’) < $str;   #tmp file
$tmpio->seek(0,0);
$tmpio > io(‘=’);   #STDERR

<,>,<< on http IO objects does GET, PUT, POST.

2 line forking webserver with CGI support

“IO::All is not your mom”

Continuing on with Spiffy.pm. Sort of a super-exporter that works with inheritance. Also some nice OO conveniences. Implicit $self, can call super to call method of same name with same arguments in superclass. field and const define mutable and immutable object attributes. Provides mixins.

Another standing room only talk, but at least I got a seat. I’m not sure who is doing room assignments, but this seems like it should have been in a larger one. I think there may be a problem of there only being small rooms and really big rooms, and no medium rooms

This talk will cover a subset of best practices used by Ticketmaster to manage a large set of servers with a modest staff. Approx. 800 production linux servers, 4 production clusters, 6000 tickets/minute (max or avg?), very bursty around onsale times. Application is divided into 24 system running on 24 distinct classes of nodes.

Operational culture of TM: automation, open source, reliability, modularity

Best Practices: everything should be active, utilize a flexible system organization, automate everything

Best Practice 1: everything should be active. Use all your hardware resources. Better to automatically decommission known systems, than to automatically commission unknown systems. Idle standby systems often decay leading to surprises when you have to failover. TM uses: multi-master Oracle replication, Active/active VRRP, clustered NAS,

Best Practice 2: utilize a flexible system organization. Operations is often the last to hear about changes. Naming of servers: <class><node#>.<product>.<cluster>.<sa>. All nodes of same class are configured identically; all automation software keys off class. All nodes have two NFS shares: /<class>/local/, /<class>/shared/. Shared typically has code, configuration, templates. Local used for logging.

Best Practice 3: Automate everything.

Configuration management: Kickstart to do OS install. per-class symlink to shared RPM repository. Overlay system copies m4 templates, by class, to root filesystem. All config files are templated. Can deploy a new server in 7 minutes.

Next gen CM (in development): always running to detect configuration drift. TT replaces m4, APT replaces symlinks. Configuration in a giant XML file with all information needed for all classes (is this a good idea?). Configuration authors must also provide auditing functionality. Define severity levels for changes which limit what time windows they can execute during.

Systems Engineering staff is 1 lead, 2 network admins, 3 sysadmins, and 1 developer.

Esther Dyson is stuck in Dallas, so it’s just Freeman and George this morning.

Freeman is mostly interested in biotech currently. He has hopes that this technology with become as common as computers, giving home gardeners, pet breeders, etc. the ability to use it in their work and hobbies.

Tim asks about possible risks

Freeman wonders can it be stopped, should it be stopped, and if so, how do you set up the rules so the it can be used for good purposes and not dangerous ones.

Tim asks George about his experiences leaving the traditional education system.

George comments that he thinks there is a false dichotomy between doing work with you hands and with your mind. On the contrary, good mind work often happens while your hands are occupied with other things. He misses the days when people learned about technology by taking apart carburators and the like. Now, shop classes use CAD instead of actually taking apart and building stuff.

Back to Freeman, Tim references the debates about nuclear technology and asks about how he views society and risk.

Freeman says people are really bad at assessing risk quantitatively. Our society is both extremely risk-adverse and at the same time highly risk-taking, because people don’t think about things correctly.

Tim quoting Mitch Kapor: “Don’t need a Department of Homeland Security, need a Department of Homeland Arithmetic”.

Yesterday was fairly good overall. I enjoyed most of the sessions I went to, and I didn’t overload myself. After the LJ talk, I ran into Geoff Young and chatted with him for a bit about an issue I’m trying to deal with. Unfortunately, I ended up rather late for the Parallel Sessions in Perl talk as a result. He was on slide 11 of 20 when I got there.

Aside: I almost ignored that talk entirely when I was looking over the schedule. That’s because at every other conference I’ve been to “Parallel Sessions in Foo” means something completely different.

After the sessions, I went to off to dinner with a bunch of veggies. We attempted to go to the Calendula Cafe, but they were closed unexpectedly. Instead we went across the street to Vege Thai. I was a bit worried for a bit when the vegans at the table started worrying that the soy “meats” might not be vegan, and I’m just thinking, my god, we went all the way across town to a completely vegetarian restaurant and you people still can’t eat anything on the menu. (N.B., I am not a vegetarian, I just live with one.) But, it turns out their fake meats don’t have egg in them, so it was all good.

Post dinner, all the Yahoo folks headed to a party hosted by O’Reilly, where we all talked to each other. Kinda lame. I wandered a little bit and chatted with a few people, but it was loud and I was having trouble focusing. There were a few people I would have liked to talk with, but I wasn’t feeling up to starting conversations.

Overview of LJ’s evolution from 1 server to ~90 currently including load balancing and distributed caching (memcached). 4 million users, 50 million dynamic page views / day.

Evolution:

• 1 shared server
• 1 dedicated server
• one web, one DB
• multiple web, one DB
• replicated DB
• more webs, more DB slaves, chaos, I/O, CPU contention

Scaling problem: every DB has to write, so eventually you lose no matter how many slaves you have.

Solution: user clusters. All user data stored in one cluster. Never join from multiple users. One global master keeps user metadata (user-cluster map)

But... still lots of points of failure. Add master-master configuration, 2 machines per cluster. Challenges: avoid corruption, unique id collisions, backup has a cold cache.

Custom-built load balancer. Using persistent connections, which helps with burstiness, guarantees connections are good. Paid users get to cut in line.

Replaced RAID with MogileFS. Lost the thread here for a bit...

Caching is important. Would be nice if it could all be handled by the DB, but there are limitations: 32 bit, application logic. Where else to cache? Mod_perl — too much memory use, bad hit rate. Shared memory — tricky, only works on one machine.

memcached is their Open Source, distributed caching system. APIs for many languages. Various machines have different free mem, provide some number of buckets. Hash request, pick a bucket. written in C, event-drived, single-threaded using libevent. custom memory allocator.

10 hosts, 28 instances, 30GB cached, 93% hit rate.

This session ended up standing room only, with me as one of the standing, so I couldn’t take notes at the time. This will be more recollections and commentary

This talk turned out to be about something a little different than I expected from the abstract. In particular, his definition of “enterprise” seemed to include only internal applications that don’t directly make money for the company (but where you can definitely lose money if it doesn’t work). I would have tended to include things like we do: high-performance, high-availibility web services clearly coupled to revenue.

Consequently, I found a number of things I disagreed with. For example, I like metadata a lot and it makes my life a lot easier. However, he seemed to advocate against config files and the like, in favor of keeping all the constants, configuration, and logic in your code.

I did agree with some things. Write tests. Keep methods clear and simple. Use OO (although, I’m not sure I’d go quite as far as he did).

A big area that I expected to be touched on when talking about “enterprise” systems was deployment and monitoring, but that wasn’t touched on at all.

Not there yet, but getting closer. Most of the design, including OO, is complete.

Pipelining operators <== and ==> make map grep max ... a little easier to read and understand. They’re lazy too!

‘fat comma’ makes pairs. Useful for named subroutine parameters, but there is also a new syntax: format $text, :indent[8] :ignore{ /\t/ }

slurp replaces local $/ nonsense

Enhanced sorting removes need to Schwarzian Transform and similar manuevers.

Real classes:

class DogTag {
  has Str  $.name;
  has Int  $.serial;
  has Str  @.record is rw;
  has Base $:posting is rw;
  ...
}

Accessor methods are created automatically. Attributes are read-only unless marked read-write. Private variables marked with ‘:’. Lots of implicit method call stuff, which I’m a little nervous about, but maybe it really is easier to read.

There is a concept of submethods which override a method, but only for that exact subclass, not for sub-subclasses. Supposedly this is good for initialization, but I didn’t quite follow the argument.

Can add “roles” to classes, which are sort of like macros that insert some attributes directly into the class. Good for component-based design. Can create a role without implementations to create interfaces.

The OSCON tutorials are 3 hours total, split in half with a 1/2 hour break. While the break is definitely good, 90 minutes is still too long to go continuously. I think it’s well known that it’s basically impossible for people to focus for more than 45-50 minutes or so. It might have been a better idea to do the tutorials in three 1-hour pieces, or maybe shorten them to two hours total.

Retrospective: This ended up being a little slow, and I suffered from trying to stay attentive the whole time. But, I think it was valuable and I’ll have a much better idea how to approach the docs when I get home and want to play around with this. I’m still not sure if it’ll be something I want to use, mostly because I don’t know what the performance characteristics of the system are.

POE is the core of a system originally designed for a multi-user game with a runtime that could be modified on-the-fly. At the core, basically an event loop that waits for events and calls handlers, which create more events, and so on. Events are queued when the system can’t handle everything in real-time.

Events, Sessions, Kernels

POE events are just lists of data from various sources.

• kernel
• event name
• sending session
• receiving session
• private data location for session
• for I/O events, socket or filehandle
• other application specific data

E.g., a session calls select_read for a socket and alarm_set, which creates events which are enqueued. When an event reaches the head, a callback is invoked (read from the socket, maybe) and this might post a new event (write to the socket). And around and around it goes.

Sessions are somewhat analogous to threads. They have a private data area which is reasonably encapsulated (but this is Perl, so if you really want to break the rules you can).

Sessions can be like “beans” or message-based interfaces. Events invoke a method on an object encapsulated by the Session. POE::Component::SubWrapper does all this for you.

You can use Session to modify event dispatch. Examples: NFA, MessageBased, MultiDispatch

Event fields are members of @_, not a hashref because that’s a lot slower. There are constants exported for field positions. Get your particular stuff with my @stuff = @_[ARG0..$#_]

Sessions only run as long as they have work to do. If they have no events or event watchers active, they get terminated. (Exception: a signal event watcher doesn’t keep a session alive by default.)

Use aliases to allow sessions to send events to other sessions. (Just friendly names, rather than unique numeric identifiers.)

Sessions can create child sessions. A child counts as work for the parent. POE::Kernel is the default child. Sessions get notification if their parent changes or if a child starts or terminates. This is not related to SIGCHLD.

Timer are events set to dispatch at a time in the future. Either absolute time or relative time

I/O watchers work like selects. Can start, stop, pause, and resume. They don’t actually perform I/O, they just watch for readiness. Your handler method should do all that.

Signal watchers look for OS signals or internal signals. Signals sent to a session are also sent to its children. If one session handles it, it’s handled for the whole tree. Makes it easier to do graceful shutdown and the like.

Wheels

Wheels are used to encapsulate common watcher patterns. They are not kept by the kernel, so your application needs to keep a reference. They create and destroy watchers and otherwise modify your session.

Filters translate data for a wheel - serialization and deserialization. Drivers perform file I/O, but basically POE::Driver::SysRW does everything you want.

Output is typically buffered in the driver.

POE::Wheel::Run runs programs and handles their I/O. Can modify permissions and priorities.

SocketFactory makes sockets with callback mechanism for success and failure via events.

Can set high- and low-water marks to detect slow connections and register events to deal with the situation. Also FlushEvent can be set to run when the buffer empies.

Miscellany

Can switch filters during a run. Useful for sending HTTP streams, or parsing headers line-by-line, then slurping up all the body.

Can switch events generated by a wheel. E.g., handle authentication, then pass off to main application.

With one and two halves tutorials under my belt, I’ve got mixed reviews. Damian’s talk was good, the afternoon ones not as good. I’m still trying to get into the mingling thing. There’s a few people I recognize from LA.pm and elsewhere who I haven’t had a chance to go say hi to. However, I did somehow end up having dinner with a big group of strangers last night.

Based on a shortage of sleep, and a review of the abstracts, I decided to skip out on morning sessions today. I was signed up for Patterns in Perl, but the handouts didn’t convince me I would get anything new out of it. I looked through the rest of the abstracts, but wasn’t real psyched about anything. So, I’m going to sit in the lobby and hack a bit and maybe even talk to some of the other people skipping.

Decided to crash a different tutorial after the break. I’m checking out the Subversion session. Sadly, the wireless is not as good in this room.

Subversion has no connection method that supports something like SSH agents. Either cache username and password, or enter it every time.

Copies are cheap, constant-time operations with minimal additional storage required. Makes branching and tagging cheap too (they are just directories in the repository). Easy to change your mind about repository layout.

Hmm... been here 30 min and we’re still essentially on that point.

svn mkdir http://... can create new directories in the repository directly. Similarly for delete. This is handy for working with branches.

status defaults to local changes. Optional to compare to repository.

cvs annotate renamed to svn blame, but also available as svn praise

svn revert to unmodified version (not necessarily the head version). Requires a target and defaults to non-recursive for safety.

Can set properties on files and directories.

merging is always complex, but Subversion is easier than CVS. svn merge -r 15:16 file2 file6 merges the changes in file2 from change 15 to 16 into file6.

Can’t put an SVN repository on a shared disk if accessing directly. In general, using a local-disk repository has issues (permissions, groups). Maybe easier to just set up Apache.

hot backups possible, thanks to Berkeley DB backend.

Speed

DBI is designed for speed, don’t worry about overhead from the frameword. As for your application, pick the right DB, design the schema right. In terms of division of work between application and DB: keep it close to the data, moving data from DB to application over network is expensive. Do more in SQL and stored procedures. But... sometimes Perl is faster than joining in the DB.

Preparing your statements speeds up execution, but can use a lot of time up-front. You need to understand how your optimizer works to get the best performance. Use hints as a last resort, best to let your optimizer do the job, since your key distribution may (will) change.

Remember the SQL cache. Use placeholders to get better cache hit ratio. But, sometimes the placeholder means you can’t fully optimize.

Selecting fewer columns is faster. Using fetchrow_arrayref improves this a lot. fetchall_arrayref is much faster for small number of columns, slower for more.

binding return columns is faster than dereferencing array.

RaiseError is faster than checking return values.

performance profiling in DBI:

> DBI_PROFILE=1 test.pl    # summary
> DBI_PROFILE=2 test.pl    # breakdown per statement
> DBI_PROFILE=6 test.pl    # breakdown per statement and method

DBI Features and Usage

connect_cached saves database handles. Good for oracle, where starting connections is slow. Safer than connecting and reusing because it handles things smoothly if the database crashes or the network goes down or whatever.

Error handling

Errors happen. Failures happen when you don’t handle errors.

RaiseError will call die for you with informative message. You can eval and examine the results in various ways. Or, you can set an error handler with $h->{HandleError} = sub {...}.

From “The Importance of Being Earnest”: Behaving well and feeling well are typically mutually exclusive. Intuitive coding is dangerous. You should be conscious of coding and only depart from guidelines after consideration.

These are D.C.’s guidelines, which might not be best for you. Most important to pick some rules and stick to them.

Highlights:

verb_noun_preposition, verb_noun_participle for subroutine names allows natural language usage. Ex., build_profile_using().

5.6 and later provide aliases from values. Lexical::Alias lets you do some additional tricks.

Implicit localization in for loops - yikes! You can’t break out of a loop and keep the value of the loop variable in the enclosing scope, because the loops is actually using a localized version of the variable which is hiding the version you thought you were using. Probably you should use a subroutine if you need this

List::Util are fast (written in C), you should use them.

You are a developer and a maintainer. So document your code.

There are lots of things that can kill you in Australia; you should go to YAPC::AU.

Don’t use prototypes. They can create non-standard semantics and they don’t work with OO.

Never return undef. You meant return;. (Returns empty list in list context.)

Don’t use bareword filehandles, use lexical filehandles instead. Then put them in braces: print {$file} $num, $name, "\n"

IO::Prompt (coming soon to a CPAN near you) provides nice features for handling user input.

Regexes:

• Always use the /x flag
• Always use the /m flag (make ‘^$’ the same as every other utility, including Perl 6)
• Use \A and \[Zz]
• Always use the /s flag
• Regexp::Auto can do this for you
• Don’t capture unless you need to
• Don’t use $1, etc. Assign immediately, or match-and-assign
• build up complicated regexes from simple units
• Can greatly speed up alternation by factoring out affixes and by using (?>...) (non-backtracking) if appropriate

Readonly, better than use constant?

croak, don’t die. Makes it easier for the user to figure out where they screwed up.

Never cut-and-paste cut code except into a subroutine. Never cut-and-paste a subroutine except into a module.

After reading various posts and comments from some of the presenters, I’m feeling some pre-conference jitters of my own.

In particular, I’m a little distressed by an attitude that seems to be moderately common among those speaking, which is something along the lines of “I wouldn’t be here if I weren’t speaking; None of the sessions are anything new or interesting to me”. There are a few possible explanations I can see.

First, the people in question could just be close-minded. I certainly wouldn’t be the first time I’ve run into people who can’t find anything interesting outside of the narrow band of work they do.

Second, maybe the talks really are mostly just rehashes of stuff you can find in the manuals or FAQs. This would be a little sad, and would reflect badly on either the conference organizers or the speakers themselves (for not putting in the effort to talk about anything that isn’t already well documented).

Third, maybe there really just isn’t anything new and interesting to talk about, at least in certain areas. I’ve heard some people make the claim that Perl is getting stagnant and boring. I’m not sure that I really buy that, though. More mature, yes, but that opens up a whole new area of interesting work to do. (I’m hoping that the POE talk, for example, fits into this category.)

Possibly, I’m just overreacting to passing comments. I’m still trying to keep an open mind at this point, although, I’m also taking a little advice from Dr. Bacchus to think about exactly what I want to get out of the convention. I don’t think I’ll go into precisely what that is just yet, though.

We flew into Portland yesterday afternoon and proceeded to hide in the hotel room from the 104 degree heat. Yikes! Around 10PM it finally cooled down enough to venture outside for a trip to the local supermarket. Oddly, we had to call down to the front desk to get a mini-fridge for the room.

The conference wireless isn’t set up yet (not too surprising), so I ordered the in-room internet access for the weekend. I’ve never actually used one of these crazy hotel internet services before. At first, I thought it wasn’t working because you have to open your browser and do some wierdness there first before you can make connections on any other ports. For a moment I was worried that they thought that "the Internet" only included port 80, but after clicking though some usage agreement nonsense, my mail and ssh opened up.

Today, I think we’re exploring the town. Tomorrow, mondo croquet, Powell’s, and maybe the brewer’s festival if the weather is cool enough.

This morning, I took the Programming Concepts test. I was slightly disturbed that I actually scored higher on this one than the Perl test, since I felt significantly less confident in my answers. (Although, I realized late last night that I made at least two stupid mistakes on the Perl test by skimming perldoc a little too fast.)

I also felt like this test wasn’t as good as the Perl one, probably as a side effect of covering a broader and vaguer subject area. In particular, the language used in a number of questions (especially related to OO concepts) was rather confusing and unclear, in my opinion. Given the debates that regularly take place on some of these topics, it seemed like a guessing game of which camp the author of the question was in. There was also at least one debugging question that seemed like a matter of opinion (debugger and breakpoints vs. isolation and unit testing).

We’re contemplating ditching our home-grown test for job applicants and requesting they take one or more Brainbench tests instead. But, of course, we need to evaluate those tests and get some normalization. So, I took an hour this morning and took the Perl test. I am now officially a Perl Master. Whoot.

The test was fairly decent, although I was surprised how few, and how simple, the regular expression questions I got were. There were only about three or four and they were almost complete just about whether you know about maximal vs. minimal matching. I got a lot more questions about symbol tables and globs than I expected. I guess maybe it focuses in on that type of thing if it decides you are pretty good (the test is adaptive), although that’s the sort of thing that I think should only be used with extreme caution, even by experts. I also got one or two weird questions where I’m half-convinced there were typos or mis-wordings in the question. Unfortunately, I didn’t think to note them down at the time.

Addendum: I forgot to mention that there were a number of questions along the lines of, what does this script output, or, why does this fail? The rules for the test say that you can use reference materials, physical or online. They don’t explicitly forbid just running the code, but it seems like that shouldn’t be allowed. I restricted myself to Programming Perl and perldoc, but it was definitely tempting to just run the code a couple of times. This seems like a concern if we really want to use this for assessing candidates. I felt like I ought to email Brainbench and ask for clarification about this.

I recently came across a link for this article: How to Misuse Code Coverage, about how not to use code coverage tools. (Interestingly it was in another person’s meditation on this idiom.) It’s pretty good, and cogently explains the vague feelings I’ve had keeping me away from certain parts of our code where I know I could trivially up the code coverage, but wouldn’t really test the important things (which, sadly, I don’t have time to write tests for at this point).

Apparently Boston’s T is planning to start searching passengers, out of increased worries due to the upcoming Democratic National Convention. The ACLU is protesting. I had a few thoughts on the matter.

• First, when will civil libertarians ever learn that the way to get support is not to say things like, “Well, you know, people do use illegal drugs, and maybe they forget that they have a joint in their backpack, and those people are going to get in trouble if they get searched.”?
• This won’t be universal searches, which would cripple the system. Rather they will be “random” searches, with all the usual concerns that brings.
• I’m not the first to say this, and I won’t be the last, but I’m really sick of hearing people say, “things are different after 9/11”.
• You really can’t say things like, “This is meant to be a deterrent. Nobody believes we’re going to approach 100% with this.” You’ve basically just told would-be wrongdoers, “we probably won’t catch you”. Deterrance only works if the chance of getting caught is high and the penalties are sufficiently severe. You have to figure that a possibly suicidal attacker is extremely tough to deter.
• Does this mean you’re not going to be able to travel on the T with a *gasp* pocketknife anymore?

There are days when I become totally disheartened and convinced that I’m the only person around here who actually tests their code. Somehow it seems that every time I take a nearly-finished project and start work on integrating it into the main code-line, I find that various tests are failing, leaving me to track down why exactly and find the person who made the change to confirm if it is a bug or an evolution that invalidates a test assumption. Since I’ve typically unit-tested my code pretty thoroughly by this point, I nearly always spend significantly more time dealing with other people’s bugs than fixing any problems I find in my code.

Given this, I have to wonder why I bother writing test frameworks, writing convenience wrappers, writing convenience wrappers to the convenience wrappers, writing documentation for the test systems, setting up nightly test runs and reports, giving talks about the test systems, coming to meetings to consult on the use of the test systems, harping on the unit tests (or, more typically, lack thereof) in code reviews, etc.

sigh...

After seeing it show up, with positive words, on enough people’s blogs, I gave in and picked up a copy of Eats, Shoots & Leaves. I’ve only finished the first chapter, but I’m a bit bothered by one of the examples.

Truss repeatedly mentions the movie title, “Two Weeks Notice”, as an example of the decline of the apostrophe. Apparently, she thinks that it ought to be “Two Weeks’ Notice”. Now, the problem here is that, as far as I can tell, the notice does not belong to the two weeks in question. On the contrary, I would consider “two weeks” to be an adjective phrase modifying the notice. Arguably, the plural is inappropriate (compare “ten gallon hat” or “five pound sack”), but I’m finding it hard to argue for an apostrophe being involved in any way.

(Skipping ahead a bit in the book, I find that she has a special rule in her list of uses for the apostrophe, for just this case (indication of time or quantity). I find this uncompelling, though, since universal application of the rule seems nonsensical. Garner seems to agree with me, although I suppose it’s possible that this is a British-ism.)

I’m feeling like maybe I’m being a little slow, but I wish I could figure out what the lesson is that Tim Bray wants us to take away from his recent post about Latency. Is is that context switches are slow, so running a client and server on the same CPU is a bad idea? Is it that Apple’s network layer is slow? Is it that Apple’s JVM is slow? Is it that Apple’s hardware is slow? Is it something about TCP/IP?

I’m leaning towards the first, but I kinda wish he’d just come out and tell us what it’s actually instructive of.

It seems that there’s two basic types of domain experts:

A) those who think they are great because they understand X, and treat anyone who displays a lack of understanding of X with contempt. They tend to reply to questions and problems with an attitude of “how can you possibly not know this?”.

B) those who think that they are experts at X so you don’t have to be. They tend to reply to questions and problems with interest, and even excitement at helping someone solve a problem they are having.

In the last week, I’ve gotten to deal with one of each, in two systems which could be seen as roughly competing with one another. I wonder if the A realizes that acting in this way tends to push people towards the other system, and could ultimately be self-defeating for him.

Actually, thinking more, I can’t really think of any cases I’m familiar with of an A really rising in an organization. They usually seem to be permanently stuck as a front-line grunt, embittered that they never get properly recognized for their abilities. The reality, though, is that the Bs really are more valuable to a company because they get around to solving problems that need solving more quickly.

(And, yes, I am aware that I have a bit of an ego, and that I can be a bit prickly at times myself. But, I’m working on it.)

I couldn’t find any evidence that anyone else has done this already, so I spent a little time hacking together a screen scraper to generate iCalendars for each of the OSCON 2004 tracks. It’s probably not perfect, and I know that a couple sessions are missing. (Cancelled sessions are semi-intentionally missing.)

Observations:

• As always, “this’ll only take half-an-hour” took substantially longer
• Apparently O’Reilly are less expert in web publishing than in print publishing, as they can’t even manage to produce well-formed HTML.
• ... much less anything resembling semantic markup
• This would have been easier with some guide to the [iv]Calendar format other than the RFC.

I think I’ve decided to jettison the popular ref $that || $that idiom from my Perl objects. To some extent, I’m just being silly because what’s put me over the edge is the fact that you basically can’t get complete test code coverage for this construct, so my Devel::Cover reports are doomed to stay short of 100%.

However, the fact of the matter is that I never make use of this “feature” of my objects. For that matter, I don’t think I’ve ever seen anyone else use this feature either. To put the final nail in, it’s a completely false analogy to C++ copy constructors. No copy is made — it’s just another default constructor.

So, no more will I use this incantation, unless I’m actually creating a real copy constructor.

I’m just getting around to reading “Beyond Fear” by Bruce Schneier. Based on the first 100 pages, I’d definitely tend to put this in the required reading category. Two quotes, one amusing, one worrisome:

Military force constitutes an attack, but a discussion of it and of military defense is, for the most part, beyond the scope of this book. If an army is invading your country, please look elsewhere.

...

Defenders benefit from technology all the time, just not as efficiently and effectively as attackers do.

The context of the second quote is fairly depressing. Of course, in many ways, technology makes us safer and more secure. However, the concept of class breaks ought to give pause to anyone who designs computer systems or other technology. As Schneier points out a couple pages later, it no longer suffices to plan for and defend against an average attacker, since a very skilled attacker can defeat a system once and distribute the method to anyone else.

None of this is really news, but it does highlight how security is much tougher in today’s world. It may also explain a bit of why most of the current practitioners don’t seem to be particularly good at it.

The practice of picking version numbers without any system behind it is seriously annoying. I’m looking at a bit of code that someone stuck a version number of 0.99 into, without any particular reason as far as I can tell. I believe that this version number is typically used as a cop-out. I.e., it should be read to mean, “I think this is done, but I’m not willing to really certify it as done, nor should you really expect me to maintain it”. Personally, I think that if you don’t want to certify a bit of software, you should give it a version number like 0.1 to make it clear that it’s alpha, unstable, and probably broken. If you put in a version number like 0.99, I think you ought to have some well-defined milestone that will cause it to go 1.0. To word it a little differenctly, if you don’t know why something isn’t 1.0, then don’t call it 0.99. If the package is feature-complete and free of known bugs, then it should be 1.0. If new bugs are discovered and fixed, you move to 1.01 and so on.

As if phones and in-person interruptions were not enough, we now have a veritable plague of channels for interruptions, which OS and application developers apparently think we can’t live without.

These days it seems that whenever I’m in any sort of meeting where a Windows user is projecting their computer screen, every 30 seconds or so, the whole room is informed that they have a new mail message (complete with sender, subject, and first line), or that some person or another is on-line, off-line, idle, not idle, or that they could be using their spreadsheet or word processor or other program better. Honestly, I can’t figure out how anyone gets any work done. I have to quit Mail when I need to concentrate just so that the small red dot doesn’t tempt me away. Maybe they just all have more self-discipline than I do.

(Windows is, of course, not the sole offender. Currently, I’m hounded by Apple’s iCal, which insists on coming to the front of my screen whenever I dismiss a reminder, forcing me to re-minimize it before I can get back to whatever I was doing. Alternatively, if I ignore or kill the pop-up box, then the iCal icon sits in my dock bouncing for all eternity, or until I give in and pay attention to it.)

Update: D. Keith Robinson just posted a somewhat related pondering

Today there is lots of discussion of Willis’s review of The Mythical Man-Month. Mostly I agree with the general consensus (at least at rating 4+) that this review is a little simplistic and even childish. The sad part is that he is so busy laughing at anachronisms that he doesn’t even recognize when they’re still true. For example,

Regarding source code documentation:

"The most serious objection is the increase in the size of the source code that must be stored. As the discipline moves more and more toward on-line storage of source code, this has become a growing consideration. I find myself being briefer in comments to an APL program, which will live on disk, then on a PL/I one that I will store as cards."

For who among us is this not true? Honestly, you just can't shut me up on cards.

However, I still hear people object to adding inline documentation to their code on the basis that it would make a subroutine longer than 24 lines, and therefore no longer fit in the now-mythical single VT-100 screen. I have suggested perhaps making one’s window a bit larger (I tend to have Emacs windows 50-60 lines high), or even that if comments were added, the increased clarity would obviate the need to have everything visible at once. Typically I have no success with this, though.

He also laughs at the idea of renting software and memory, despite the fact that many companies do lease hardware, and software subscriptions and support contracts are also still current concepts.

I’m browsing Ticketmaster and it’s really annoying me because of a combination of two (mis)features.

First, there is this bot-blocker thing, where you have to type in a word (or some word-like string) that has been obscured. The problem is that I guess the bots are getting better, so the obscuring is getting more extreme to the point where it’s getting hard for me, a human, to actually figure some of them out easily.

Second, for some reason, the “look for tickets” page lets you search for ticket prices / locations that are sold out. So, I’m doing a lot of loading slow pages and entering silly words hunting for tickets which mostly don’t exist.

Since there’s actually some chance of someone who works at TM reading this: Hey guys, fix this!

I think it would be a good thing if more people read The Economist. First off, they would be a lot more educated about the state of the world. But, also, they would have a better idea of what the rich really think about them. Or, at least, that the people who write ad copy targetted at the rich think the rich think about them. Recent examples, paraphrased:

• Virgin Atlantic First Class: helping you forget about those annoying people who work for you.
• Intercontinental Hotels: because we understand that you deserve a hotel room bigger than most people’s houses
• Citigroup Family Advisory Practice: let us ease the burden of being born rich

The GNU organization has got to be the most un-cool group in existence. I cannot believe that the Emacs manual has moved yet again. Oddly, my bookmark to the Emacs Lisp Manual still works.

Just completed registration for OSCON. Towards the end, the form sure gets nosy. Observations:

• Apparently “I just work here” isn’t in their conceived of job positions. I went with “Staff”.
• The list of industries is also a little odd, as there is nothing that screams “software shop” to me. I went with “Manufacturing/Design (IT/OEM)”.
• It’s kinda fun to click the “we are big, we spend lots of money” buttons
• I wonder if most companies are really comfortable just telling whoever what sort of software and hardware purchases they are planning. I went with “No planned purchases”, partly on the idea that maybe they mean me, personally.

Apparently there is some code reuse going on at O’Reilly, since you can enter a credit card expiration date of 2003.

I’m not really sure why we have travel agents. Because I’m staying a few extra days on each end, I edged over the limit for the conference rate on the hotel. However, our travel desk tried to tell me that the hotel was sold out. A quick call to the hotel set that straight, though. It just bugs me that I have to be the one to make that call and figure it out, given that I’m required by corporate policy to use the travel agency.

Just check out this exchange:

...

ASHCROFT: I'm not in a position to answer that question.

LEAHY: Does that mean because you don't know or you don't want to answer? I don't understand.

ASHCROFT: The answer to that question is yes.

Only a programmer could give that answer.

Or a big huge weasel of a lawyer.

[This is a modified version of something I blogged at work, somewhat edited to protect the guilty]

There are perennial discussions about how to encourage software reuse, mostly with the unstated assumption that developers are somehow opposed to the idea and need to be convinced to do it. Occasionally, people even suggest that we ought to give bonuses or something similar as incentives.

This seems to be a fairly bad approach to encouraging code reuse. In fact, a fair number of respected people in the software development field (example, example) consider bonuses to be a bad form of incentive in almost all situations. Beyond the question of whether this is an effective way to motivate developers, without a lot of careful thought, a system of directly rewarding the producers of reusable code is bound to create a disfunctional work environment as it destroys alignment between the goals of workers and their immediate supervisors.

My belief is that left to their own devices, the majority of developers want to produce good code which other people will use. There is nothing more depressing than spending a lot of time and effort on a project that no one ever uses. Conversely, just knowing that people are using and appreciating your work is a great incentive (assuming that basic needs like food, shelter, and toys are being met via salary). The obstacles to producing reusable code are almost entirely at the low- and mid-level management level, not at the developer level. This is the group which must be encouraged, through reasonable schedules and vision at the CTO/CIO level, to create the code reuse environment.

The only major exception I see to what I’ve just said is documentation. Most developers don’t like writing documentation and don’t do it well, even if given the time to produce otherwise excellent software. Fortunately, this problem is fairly easily solved by just hiring writers.

The flip side to all this is encouraging people to actually reuse the code which has been produced elsewhere in the company. In my opinion, this is a much bigger problem. Naively, the situation here is reversed in that managers should see reusing components as a solution to their schedule problems, while developers tend to resist for a variety of reasons. More realistically, there is probably resistence at the manager level due to the increased cross-team interaction required in finding and validating appropriate components, and enhancing them if required.

On the developer front, the biggest issues are probably those of knowledge management (finding appropriate components to use) and documentation (finding out how to use an ostensibly useful component). The documentation problem is relatively easy, as I pointed out earlier. Unfortunately, knowledge management is hard. Really hard. A system for cataloging software components has to go well beyond simple keyword searching. It needs to be cognizant of interface, semantics, performance and scalability metrics at a bare minimum. I don’t really know what’s out there at the moment. And, it’s slightly possible that the answer here is actually just as simple as “hire a librarian”. But, I sort of doubt that library science is as evolved in this area as we would need for this to be a routine thing.

We also have to admit that there is the social aspect of developers thinking that it’s less fun or somehow a display of weakness to use other people’s code. Some of this is legitimate in that it can be dramatically less fun when the code doesn’t perform as advertised (or doesn’t advertise how it performs). Additionally, various propagandists tend to make it sound like the job of developers will ultimately get reduced to hooking together a bunch of preexisting components with no original work involved. People need to be reminded and reassured that there is going to be a continuous need for new components and that someone needs to build those. Reusing other components then lets you focus on the core of the new component you are working on, and gives you the time to make that system excellent as well.

Just a couple quick links:

• Very cool, we need this for the US: TheyWorkForYou.com (from Simon Willison)

• Very scary: pain beams (from jwz)

I finally got about to scanning in some photos from a visit to the Huntington back in January.

(more)

One of the things which has always bothered by about XP is the “my way or the highway” attitude in some of the literature. What I’m talking about is the statements along the lines of, “I don’t care if you want to program some other way, just don’t do it on my team”.

Yesterday I was talking with someone who has spent most of the last two years working in an XP shop. It came out that in the 3 months after they started using XP, they experienced 50% turnover in their development team. He didn’t seem too bothered by this, but I think that most of the respected experts in software development process (say Brooks, or DeMarco and Lister) would say that you’d have to be insane to adopt a methodology (or take just about any action) that results in this level of turnover. The costs to the organization are so immense that it’s almost guaranteed to be a bad idea.

Can I just point out how incredibly lame it is that this is the top hit for my name on Google?

Yesterday I had one of those epiphany moments when I figured out why corporations might be interested in RSS, and why it’s more than just blogs. I spent a little time last night trying to explain to some of our platform guys where I thought we could get some use out of it. I’m not sure I did the best job of explaining my thoughts, and there’s definitely some skepticism about why RSS rather than some other format (because it’s there, I think, and there is a large selection of aggregators available), but it might have gotten people thinking about it a little. We’ll see.

This technique is just awesome. (link courtesy of jwz)

Tim Bray expresses some reasonable thoughts about what’s been going on in Abu Ghraib. Of course, this is exactly what the Geneva Convention has always said. “We were just following orders” or “we were under a lot of pressure” or “our training didn’t cover this” are not valid excuses.

At this point, even The Economist is calling for Rumsfeld to resign over this. I’m not a fan of Rumsfeld by any means, but I do think that this might still be a little premature. It is possible that these abuses were known all the way to the top. Certainly some statements from the government’s lawyers in the Guantanamo cases could be interpreted to support the idea that torture was officially sanctioned procedure. And, I don’t believe for a moment that those soldiers were acting completely on their own with no knowledge or sanction from superiors. However, I think a reasonable investigation into what happened is warranted.

One of the primary arguments for someone at the top taking the fall for this seems to be that it would restore US credibility in the international community. After the refusal to support the International Criminal Court, this will look really bad unless the rest of the world is convinced that this issue is fully investigated. Of course, what would look even better than Rumsfeld resigning would be for us to admit our fallibility, put our backing into the ICC, and turn this whole matter over to the international community to handle.

I’m officially not a fan of Yahoo! making paystubs only availible electronically. I’m sure this policy is great for people who don’t actually keep any financial records, but it’s a pain in the ass for those who do. Of course, I can go to the website and print a copy, except that first I have to remember to do this, and then I have to deal with the fact that the “printable version” inexplicably doesn’t really print nicely on standard 8.5x11 paper.

The final consequence of this is that it’s May and I still don’t have Quicken correctly set up to auto-record my paychecks. Grumble...

I just finished some changes to make and keep this blog XHTML compliant. Fixing up the original validation failures was pretty easy, but illuminated some shortcomings in the way I'm writing my entries. Most notably, using the blox plugin just causes trouble. Really, I wanted to just make my entries valid XHTML from the start, with the help of Emacs PSGML mode.

The only real hitch in doing this is dealing with the title line. Right now I'm handling this with a real simple plugin that I whipped up that lets me wrap the title in any dummy markup, which it then strips out. So, now my entries are valid XHTML fragments, which is something PSGML mode deals with just a little bit of configuration.

I'll probably spend a little time to clean up, comment, and generalize the plugin, then contribute it to the plugin registry. Version 2, if it ever happens, will try to actually validate each entry as they are generated.

The problem with economists is that they think that money is the only thing that matters

A talk on Weds. mentioned that the expected payout betting on a 200 to 1 longshot at the race track is miserably lower than betting on the favorite. A few people suggested that these people might be deriving some benefit from such a bet other than the incredibly rare payout, but the speaker completely pooh-poohed that idea.

He also talked a little about the fact that people are really bad at understanding how rare some event are, and that they would be better considering probabilities below a certain level as just zero. An example is the lottery. On the other hand, for most people, a lottery payout is an effectively infinite amount of money, so I'm not sure that would work.

Overall, though, it was a pretty interesting talk, and it introduced us to Tradesports where you can bet on all sorts of interesting things.

Martin Fowler recently asked IsAgileForAll. He asserted that:

When a new technique or tool appears, it's usually tried out first by higher ability developers. This is quite a natural response. Early adopters have to be those who are more thoughtful and caring about their profession.

Now, this may be true that there are thoughtful and skilled people who care deeply about the software development profession among those who have adopted XP and other agile methodologies, but it also seems to me that there are a number of people who have jumped on the XP bandwagon because it is cool, fashionable, or because it seems easier and more fun than other methodologies. I don’t see any reason to think that these people should have much overlap with the set consisting of the best developers. (Nor, do I suspect that the outcomes of projects involving these two groups should be particularly similar.)

To be fair, this is sort of irrelevant to Folwer’s point, which is that it is hard to predict the success of a methodology in the general population based on the experiences of early adopters. Almost certainly there will be some bias in that sample population. However, I do think that Agile proponents may be misleading themselves about who they are appealing to. The real worry is that they may discount failure stories with excuses like “They didn’t do it right”, or “Their developers were too junior”.

I swear I’m going to kill the next person I hear use the term SWAG.

I want a new OS
One that won’t make me hate
One that won’t make me reap my child
One that won’t make me wait()

I want a new OS
One that won’t keep me up
One that won’t make me trap
SIGPIPE or SIGHUP

A couple people at work subscribe to what I like to call the solispist interpretation of quantum mechanics. (They like to think of it as anthropism.) In short, the claim is I will live forever, because the wavefunction will keep collapsing in ways that keep me alive.

After this past weekend in Vegas and tonights poker game, I'm almost starting to feel this way about myself and gambling. I suppose we should let N get above 10 before coming to any firm conclusions, but you can't win every time, right?

We took a trip up to SF this past weekend to visit a few friends, and relax a big over spring break. On Saturday, we got up "early" and headed to the farmers market at the Ferry building. Outside at the Embarcadero, I snapped this picture of a sculpture in front of one of the buildings.

(more)

I've been experiencing a little vertigo here and there for the last week.

This afternoon, I was interviewing someone and in the middle of asking a question that I've only heard asked once and hadn't prepared to ask (a bad idea, but my usual questions were hitting dead-ends) my head started swimming. I managed to finish asking the question, but then for the life of me couldn't remember which was the right answer. Fortunately, the world settled down before the candidate finished answering, so I managed to recover okay.

On the topic of interviews, about two weeks ago, I had the somewhat unsettling experience of a candidate referring to something I wrote here several months ago. I guess that turnaround is fair play in this world of companies ... uh... Yahoo!ing prospective candidates, but it really put me off-guard for a moment.

I’ve just swiped this bunch of emacs goodness from Tim Bray. I had to compile and install Emacs 21.3 to get things working, but it seems good now. (Actually, I think I could have hunted down a bunch of elisp packages instead, but it seemed easier to just grab the upgrade that has them all already.) I’m a little annoyed that RedHat doesn’t have an RPM version yet, but whatcha gonna do?

The only oddness is retraining my fingers to double hit the apostrophe key. Interestingly, I seem to have immediately adjusted to the change for double and single quotes. I think this probably says something interesting about linguistics and grammar, that I can un-overload the apostrophe symbol with no trouble at all.

That’s actually Orbitz’ motto. “Most”. Apparently, I have found the fares which are not easy. Trying to find a reasonably priced hotel in Vegas for next weekend, and their search keeps returning different results on identical searches. Then, when I actually click on “book it”, they don’t go to a reservation page, but instead the same page will reload or I’ll get sent to the front page.

I called their support line, and after spending half an hour on hold, I got a guy who claimed that that’s normal behavior when a rate has sold out. When I asked why they didn’t stop showing a rate that wasn’t availible, he said that the hotel has to update it before it’ll disappear. Now, it seems to me that something in their system knows the rate isn’t availible, in a seriously broken fashion, so I don’t really understand why their search engine can’t figure it out.

Okay, I realize the proper geek thing to do is to be in favor of it, but this ruling is just bollocks.

I cannot see a real difference between a library that places a photocopy machine in a room full of copyrighted material and a computer user that places a personal copy on a shared directory linked to a P2P service

Sigh... a reasonable analogy would be a library giving public access to a high speed printer connected to digitized versions of all their copyrighted holdings. Which you don't see them doing, as it would be a clear violation of copyright law.

The Village Voice had two good articles recently:

State to Church: I Want a Divorce

I'd Leave the Country, but My Wife Won't Let Me

The first it pretty dead on my currently feelings about marriage. That is, that the state should get the hell out of it. The whole gay marriage mess is due largely to the connotations various people attach to the word "marriage". So, let the state offer civil unions to all takers, and attach Social Security benefits, hospital visitation, 5th amendment protections, etc. to that; and let churches offer marraiges to whoever they please, with no civil implications.

Going even further, and recognizing this is completely politically impossible, toss marital status from the tax code. If the goal is to reward family formation, then tax breaks for children are sufficient. But there's just no reason why a married person with children ought to come out thousands of dollars a year ahead of a "head of household" with children.

It's very quiet at work today, although there doesn't seem to be any significant number of people missing. This ought to be great except that it's so unusual that instead of coding away happily, I keep looking up every 5 minutes and thinking "Why is it so quiet around here?". I guess you can't win.

The photo bug has definitely bit me again. This is partially helped by the resurrection of my digital camera, but it had definitely started before that. I've taken a few decent pictures recently, and I've also been scanning in some older ones.

(more)

I'm simultaneously happy and annoyed. After months of non-functionality ("Zoom Error"), my digital camera spontaneously works again. I was about to see if I could improve things with the aid of my trusty blower brush (err... my new trusty blower brush, which has recently replaces my not-so-trusty-after-all blower brush which went missing), but now there's no problem.

I've been working on moving my various web pages over to CSS layout, and the non-DWIMminess is pretty frustrating (not that I am by any means the first person to observe this). Most recently, I was working on this page, and my first pass ended up looking like this instead. It's a sort of cool effect, but I really don't understand why the containing div doesn't get the height of the photo. So, as a result, I'm forced into pixel layout on the main div, and it probably doesn't look right on IE because I didn't bother with the box hack.

Update: apparently I needed to read Eric Meyer's article on this topic.

In Chapter 28 of Peopleware, they claim that the following managerial techniques tend to induce teamicide:

• annual salary or merit reviews
• management by objectives
• praise of certain workers for extraordinary accomplishment
• awards, prizes, bonuses tied to performance
• performance measurement in almost any form

I don't totally disagree with most of these, and I definitely think that spotlighting individuals is usually a mistake, but I'm confused by the first point and some of the others to a degree.

It seems to me like they are saying that raises and salary should be decoupled from performance. Interpreted broadly, this might also include promotions. This seems like insanity. Wouldn't it also seem to be teamicidal to let your star performers feel like the only way to move ahead is to shop themselves around to other companies?

User Friendly has in their "Real Tech Support Stories" today:

James: Person calls and tells me that the DSL connection is not working and wants to know why. I see on thier account that they have purchased a wireless router from us and ask how many lights are on the router and the modem. Customer says "How am I supposed to know that?!" "By looking at the two products." Heres the kicker, you know what's coming... " I could not possibly see those things, they are in Pennsylvania and I am in Arizona."

The problem with this is that I've done this. In fact, many of the times that my DSL goes down, I call support from my office. I'm not sure what else I'm supposed to do, since they don't really have any meaningful support after-hours.

Using unsafe signals in Perl 5.8.1 and later can have some bizarre consequences. For example, this code:

my $time;

$time = `date +%H:%M:%S`; chomp $time;
print "BEFORE [$time] ( $/ = ", unpack('H*', $/), " )\n";

eval {
  local $SIG{'ALRM'} = sub { die 'timeout' };

  alarm(2);
  my $out = `sleep 5`;
  alarm(0);
};

$time = `date +%H:%M:%S`; chomp $time; 
print "AFTER [$time] ( $/ = ", unpack('H*',$/), " )\n";

$/ = "\n";
my $foo = "blah\n"; chomp $foo;
print "FOO [$foo] ( $/ = ", unpack('H*',$/), " )\n";

Produces the following:

%% PERL_SIGNALS=unsafe ./chompfail.pl
BEFORE [13:38:49] ( $/ = 0a )
AFTER [13:38:51
] ( $/ = 0a )
FOO [blah] ( $/ = 0a )

With safe signals, it's fine (but doesn't interrupt the sleep.) Also, with perl's sleep function it works fine. And, in the perl debugger, it seems to work fine.

I'm not sure what's going on here (since it isn't as simple as $/ being set strangely), but I think the clear lesson is that unsafe signals are to be avoided if you don't want to risk undefined behavior after the signal, which you probably don't if you're running a long-lived server process.

The place I usually get my film developed and printed has always sent it to Fuji. However, the last roll, they asked me which I wanted, and for some reason I went with Kodak. I'm not sure why. I think it was something silly like Kodak film, Kodak developing. (Or, more like specialty Kodak film, Kodak developing, since I've sent many rolls of Kodak Gold to Fuji.)

At any rate, it seems to have been a mistake. The quality of the prints isn't any different, but they charged me more and didn't sleeve the negatives or give me one of the "Free 8x10" coupons that Fuji always includes. (Also, extremely minor, but Fuji prints the date and frame number larger on the back.)

Proponents claim that Prop. 58 makes sure that the deficit refinancing bond is a 1-time thing that can never happen again. The problem with this logic is that the existing state constitution already forbids it from ever happening. 58 is an attempt to circumvent the clear intent of the constitution that long-term borrowing be used only for physical works. In no way does it realistically make this a 1-time event; on the contrary, it sets a precedent for doing exactly the same thing again if we get into trouble.

At the same time, no one (well, no one except Gray Davis) has seriously proposed to address the structural problems in the state budget. Arnold's "tough cuts" so far total $1 billion on a $15 billion deficit. How are we actually supposed to believe that we aren't in for another massive deficit next year?

Perl is supposed to recognize the UTF-16 encoding, with a byte order mark to determine the endianness. Unfortunately, if you try to read in a file with this encoding, it expects to find a BOM at the beginning of each block it reads in, and bails out on you.

Well, fine, so most of the time you actually "know" the endianness, so just use, say, UTF-16LE instead. Except that now you have that BOM on your first line. Maybe you figure this is no problem since you were already doing some validation / cleanup on the data and had a s/^\s+// in there. Except, Perl doesn't include the byte order mark (aka "ZERO WIDTH NO-BREAK SPACE") in its whitespace class, so now you have to do something ugly like s/^[\s\x{FEFF}]+// instead.

Bleech!

I'm starting to get really sick of the anti-database cult. Another of the new guys asked today why our whole system isn't just one lookup in a giant hash table.

Get off the bandwagon already! Just because Google supposedly doesn't use databases doesn't mean nobody should. If you want to do whatever Google does, go work for Google already!

First off, not everyone has the same system requirements as Google. We don't even have the same requirements as Google. Our primary business is not their primary business, and vice versa.

Second, of course they use databases. It's a good sign of the mindlessness of this cult that when someone claims Google doesn't use databases, and I reply, "You mean they don't use RDBMSs?", they usually come back with "What's an RDBMS?".

Third, while I'm willing to believe they don't use an RDBMS for their algorithmic search, or even for the front-end of AdWords, I simply do not believe that they don't have any back-end systems which are at least morally equivalent to a relational database.

A moment ago, while trying to find the link to this amusing video, I went to search on "really good". Google was not at all helpful, while http://search.yahoo.com happily displayed as result #3, RatherGood.com. Now, if that isn't the search nirvana of finding exactly what you meant, I don't know what is.

From Chapter 12 of Peopleware:

There are some prevalent symbols of success and failure in creating a sensible workspace. The most obvious symbol of success is the door... The most obvious symbol of failure is the paging system.

• Peopleware, by DeMarco & Lister
• The Design and Evolution of C++, by Stroustrup
• Learning Python

"D&E of C++" is our new ALACPP book. Python might be the new language I learn this year. And, I've been meaning to read "Peopleware" for a while.