Kevin's Weblog

I wish I’d brought my camera to work today. It is scary in Old Town, and not just the people camping out. Many of the businesses are boarding up their windows, as if they expect riots to break out.

I wanted to buy a Scaldis Noel glass from Lucky Baldwin’s, but apparently they hide all the breakables from the ravaging hoards, so I have to come back next week. I do wish they would put alcohol percentages on their menu, though. 12% !!! I might as well just go home for the afternoon, except that I probably shouldn’t drive either.

The death toll is now over 100,000 in Asia, and over a million people displaced from their homes. This could still rise substantially from resulting disease outbreaks. The most convenient way for many to donate money toward helping with this situation is though the Amazon Honor System, although they will only process payments up to $50. [Update: The limit might be higher than that, as the current average contribution appears to be about $60, and that page does talk about tax receipts for contributions over $250. However, I did get told that they couldn’t process my donation and the FAQ page says $50 is the limit.] If you want to contribute more than that, you need to go directly to one of the aid organizations, like:

• International Red Cross
• American Red Cross
• Doctors Without Borders

It can be difficult to decide what is the right amount of money to contribute in these situations and it is, of course, a very personal choice. For myself, the method I used was this: over the last month, I got some unexpected or unbudgetted money in the form of bonuses and stock option exercises. I took the post-tax amount from these, calculated 10% of the total, and rounded a bit, then donated that amount. For those fortunate enough to have gotten some kind of year-end bonus, I think this is a pretty reasonable sort of method.

Why is it that I totally hate project planning at work, but for a dinner party, I’m willing to write page upon page of inventories, tasks lists, and timelines at a level of detail that any sane person would have fired up MS Project to handle? I think this reflects sometime about the relative maturity of the two disciplines. Either that or it reflects that I’m a freak.

On the plane ride back to Maryland yesterday, I read The One Minute Manager Meets the Monkey. It’s a pretty fast 130 pages, but it definitely seems to fall into the category of management books that have a single insight that they simply repeat over and over until they fill enough pages to justify calling it a book.

In this case, the general insight is that you will be happier if you don’t take on tasks that aren’t (or shouldn’t be) your responsibility. If you are a manager, your reports will be happier too if you don’t take on tasks (monkeys) that they ought to be doing instead. A few techniques for getting out of the habit of picking up monkeys, and that’s basically the book.

Anyways, I’m not complaining too much because I needed something to read on the plane, and this book is a lot more compact than Quicksilver, which had to be relegated to my carry on.

Joel has an interesting article today on pricing software. I don’t think that there’s anything shockingly original there, but it does bring a lot of stuff together on one page. I would have liked to see some discussion on open-source / free software added (beyond the comments on the reality of demand curves), but I understand that Joel isn’t really in that business.

The local paper reported a couple days ago that “Social Security [is] a Gordian Knot”. I’m wondering if they actually put any thought into that metaphor, or if they just wanted to say “a big mess” and thought it sounded cool. (I didn’t actually have opportunity to read the article.)

The paper does have a conservative slant, so it’s possible that they really did mean it, as the popular opinion in that world does seem to be “in order to save Social Security, we have to destroy it.” Of course, the metaphor is completely false, as there are obvious and simple ways to unravel the problem (which isn’t actually anywhere near as bad as some would have us believe). Either of a small increase in the cap on Social Security deductions or a small increase in the age where benefits begin being paid out would be sufficient to get us over this hump. (Or a reduction in benefits, but that’s even more dead-in-the-water than either of those.) But, I suppose that that doesn’t make for a good headline.

It looks like the issue of inline advertising in RSS feeds is getting a lot of attention lately, with posts from Jeremy and John Battelle. I’m sort of surprised that Jeremy seems so hesitant about the concept. Personally, I’m really excited at the prospect of us getting into this area. But then, I work for the money-making part of the company so I’m used to viewing the web in a more commercial way than most. I also find the technical challenges of the problem really interesting.

There are some things I worry about, though. The main one is how various aggregators will react to this. I can imagine that some aggregators without another revenue model might want to put contextual advertising of their own beside feed content. The risk here is that the total quantity of advertising does go over some line and people rebel. Another thing that might happen is that aggregators might try to block ad content in feeds. Particularly if this is combined with them adding their own ads, I could see this leading to some nasty legal fights.

The crashes are back. They started slowly last week and have worked up to 4 or 5 a day this week. I guess I was too quick to blame Sailing Clicker, although I suspect that it was tickling something in the kernel. Briefly this morning, I thought that somehow something in fetchmail was to blame. My fetches had been failing since last Tuesday when my password got expired. After I corrected that, I was crash-free for the rest of the day, but after I got home I got several more.

At this point I’ve pretty much concluded that there is just something wrong with the image they set up for the pilot program at work, and a reinstall from scratch is the only way to save my sanity. I do have some evidence that they tinkered with things they probably shouldn’t have, as various programs I’ve attempted to compile have failed because they couldn’t find things like libmalloc. So, this evening I did a full backup. Tomorrow, I’ll start in on the install.

I’m not much of a sports fan, but I haven’t been able to avoid coverage of the Ron Artest controversy. I just had to wonder, is there any other profession where you could, after any provocation, dive into a crowd and start a fight by hitting one of your customers, and not expect to be fired? I guess I just don’t have much sympathy for these guys claiming that their suspensions are too harsh.

(I guess perhaps movie stars might also enjoy this level of privilege.)

I use two Firefox extensions: the Bloglines Notifier, and the Bookmarks Synchronizer. They are both cool and very useful and I couldn’t live without them. Now, with the 1.0 release, the Bookmarks Synchronizer became incompatible. However, I did find the new version after a couple days. However, when I installed it, the Bloglines extension stopped working.

Fortunately, when I uninstalled both, then reinstalled them, they both returned to functionality. However, this does seem to indicate a bug, and I’d have to guess that it’s a bug in Firefox. (In fact, even if there is a bug in one of the extensions, the fact that installing one extension can cause another to completely stop working seems to suggest a design bug in the extension framework.) At any rate, I filed a Bugzilla bug, but they are insisting that I should sent it to the extension authors instead. However, it seems that neither one wants to know about bugs since there is nothing on their webpages to let me know their email addresses. (The Bookmarks author invites me to send him email via some forums website which requires registration to do it, but WTF?)

Come on people. If you’re going to release software, please make it easy for people to contact you.

Tomorrow we’re heading off for a long weekend trip up the coast. We going to a tiny little town that we are told has nothing to do, where we intend to do just that, perhaps with a jaunt over to Paso Robles to acquire a couple bottles of wine to accompany us in our nothing. It’s going to be great.

I have a new reason to dislike the Electoral College. Not because it overrepresents rural areas. Not because the winner-take-all allocations distort the popular vote. Today it is because of the realization of how much it contributes to the appearance of deep division in our country because of the binary red state-blue state maps that come out of it. Well, the next time someone tells you that some state is red or blue, you can tell them that they’re all lavender

Even more interesting in some ways is this county-by-county map. What I love about this is that it shows that there are more bastions of solid Democratic support in the south than on the west coast or the northeast.

Update: one more variation, again vectored by jwz, this time a purple cartogram.

This worries me (from Will Saletan):

Kerry led 51-49 in the Ohio exit poll this afternoon. But he also led 51-49 in the Florida exit poll, and we've seen what happened there. Nationwide, the exit polls had Kerry up 51-48. But with 80 million votes counted already, it's Bush who has a 51-48 lead. So at this point, the exit polls are at best meaningless. Or worse, if you're a Democrat, the six-point gap between what the exit polls predicted for Kerry nationally and what the returns show so far means that in Ohio, a two-point lead for Kerry in the exit poll foreshadows a Bush win by as many as four points.

I worry about whether provisional ballots are actually going to be counted, not just in Ohio. In the first nationwide use of them, it seems critical that we count them all. Many people expect them to swing strongly Democratic, and even if it’s not enough to save Kerry (and it probably isn’t), the data is important.

I’m also pretty upset with Kerry’s logic of it being better for the election system to concede. Why is it not better to actually count all the votes? Doesn’t this just tell all the people who might be first-time voters, or move frequently, or whatever reason for casting a provisional, that their vote doesn’t really matter? Do we not want them to vote next time?

I must say, OBL’s new tape is a mindfuck of the highest caliber. Total world series of geopolitical poker. I have a feeling you’ll see people second (and third and fourth) guessing this for the next three days and then some. As in poker, it’s probably best to just play the cards you’ve got and not try to figure out what type of psychology your opponent is using. However, I would be surprised if there isn’t a noticable jump, probably in Bush’s direction, in the polls over the next day or two.

We’ve got two props on the ballot involving taxes. 63 hits $1M+ earners with an extra 1% to pay for mental health care, and 67 adds additional fees on telephone services to pay for EMS services.

63 is designed to be popular (or populist) with it’s sock-the-ultrarich approach. The won’t-hurt-me argument almost won me over, but on further thought I come down against this. The first point is that CA income taxes are already extremely progressive and high. The rates ramp up quite quickly and those with more than $50K or so of taxable income see a 9.3% marginal rate. When I consider this and ask myself if I think that the tax structure ought to be more or less progressive than it is, I find that my answer is less. Beyond this argument, there is also the fact that many rich people already arrange to “live” in states with lower taxes. I suspect that the opponents are correct that this sort of vindictive taxation will just make that phenomenon worse.

I have some reservations about 67, primarily because of the slightly dishonest way that it piggybacks on the 911 service fee, and because I’m generally opposed to the way that the state has choosen to tightly couple specific revenue sources to specific expenses. However, I do think that it is critically important to keep emergency rooms open. While it is true that a substantial amount of this money would go to private hospitals, the fact is that they are required by law to provide emergency services to all individuals even if it is unprofitable. It’s completely reasonable in my might to provide subsidies to encourage companies to actually do this, rather than closing their emergency rooms completely.

Holy crap! The Economist just endorsed Kerry. Granted, this is probably the right way for them to go given their biases and values (responsible fiscal and international policies, and lack of government intervention on social issues), but it comes as rather a surprise given how down Lexington has been on Kerry and Edwards.

I’ve been meaning to comment on the propositions on the CA ballot for a while, but haven’t had a chance to really dive into them until this week. Hopefully, this will be the first in a series of posts covering most of them.

The two election related props seem like pretty straightforward choices to me. 62 is a clear NO. Fewer choices on the general election is bad, not good. In addition, anything that allows crossover voting without other election reforms opens the door to all kinds of gaming of the system.

Prop 60 doesn’t actually change anything about the current system. However, because these two propositions are in conflict with one another, it is important to vote YES on this one in the case that a tie-breaker is needed if both pass.

(Amusing note: The “Independent Voters Guide” that I got in the mail today urges a NO vote on 62 because if it passes there could be no Democrat in the general election for some districts. Uh... aren’t we supposedly independents? And isn’t this California, where the opposite seems more likely?)

There’s been a couple interesting articles at The Ninth Bit in the last week. Megan explains Why She is a Libertarian which tends to agree with my libertarian leanings. (Although I have to say I would never label myself a capital-L Libertarian.) I’m glad to see that I’m not the only person sympathetic to most of the libertarian platform, but uncomfortable with the gun-whacko-ness.

I found the link to this article, Morlocks vs. Libertarians interesting. Somehow I’d never considered that someone might be seriously trying to pull a Nader from the other side. I have noticed, though, that in polls that include Badnarik he tends to do at least as well as Nader.

I do have to say, though, that this “taxation is slavery” stuff is crap. You can renounce your US citizenship and move to a country without income and property taxes. As long as you are free to leave, you are not a slave, but a willing participant in a social contract.

Can someone explain to me exactly what it is that credit card companies mean by “close an account”? Because it clearly isn’t what I mean by that phrase. I’ve closed three or four credit card accounts in my life and they all still appear as active on my credit report. Furthermore, in all cases, charges have continued to show up on the accounts after they were supposedly closed. Usually this is things like regular bill payments, but in two cases they were fraudulent charges.

Yesterday I came home and found a letter claiming that my Sears card, which I haven’t used in 7 years and which I canceled nearly two years ago, had a $55 balance that had been sent to collections for non-payment. The number of things wrong with this is just amazing. First, the account is closed. Second, this is the first I’ve heard of it, despite the charge in question getting made back in May. But it gets worse. They claim that in April, a $9000 payment was made to the account to cover the balance at that time! At this point, I’m starting to suspect that their database has gotten corrupted somehow. Fun.

The phone conversation started out really interesting. The letter didn’t actually include the account number. So it took a couple minutes of going around in circles before I could get them to even start dealing with the problem. “I have a letter from you claiming I owe money on an account that’s been closed for 2 years.” “What’s the account number?” “I don’t know, I closed the account 2 years ago.” “Well, I need to know the account number to help you.” “So, what am I supposed to do about this letter?” “What’s the account number?” ...

Look Apple, it really doesn’t make me feel any better when my computer crashes that I get a really pretty translucent screen of death, instead of an ugly blue one. In fact, it pisses me off that someone spent time making that happen instead of fixing the fucking problem that caused the computer to crash in the first place.

While I’m on this rant, I’d like to point out a big problem in the whole “The Web is the Internet is the Computer” thing, which is that my local desktop apps are pretty good about periodically saving my work without me having to do anything. However, when I’ve spent the last hour typing into a little box on a web page (which is a crappy way to do text editting to begin with) and then the network goes down or someone trips over a power cord or one computer or another crashes, I lose all my work.

As if to prove my point, the computer just crashed again before I could upload this to the server. However, I didn’t have to retype a thing.

Update: After 5 crashes yesterday, I gave up for the day. This morning on a hunch I uninstalled Sailing Clicker and so far today I haven’t had any problems. I’m somewhat disappointed as I think it’s a neat piece of software and I was looking forward to playing with more of its features, but at this point I haven’t even given a presentation since I installed it and clearly I can’t afford this level of instability.

Charitable organizations and I seem to be having a hard time of it this year. Yesterday, for the fourth time this year, I declined to make a contributation that I normally would have. All I really ask is that they not annoy me, but that seems to be an unreasonable request.

1. I did not renew my membership to MOCA this year. This was mostly because I hardly ever go. I hardly ever go because whenever I consider it, I go to their website to see what it currently on display and I rediscover how completely unusable their it is. I’m not just talking about bad layout or that sort of thing. Rather, they put all their content in this wierd custom scrolling box which has the unfortunate property of scrolling diagonally rather than vertically, so you can’t read past the first page.
2. Consequently, when I am in a museum mood I usually end up going to LACMA. Except that I decided not to renew that membership this year, mostly because they (or at least their security guards) are extremely un-family friendly and I end up getting in an argument with someone nearly every time I go. They have also been misleading in their marketing of various special exhibits.
3. I didn’t renew my KPCC membership because I was disgusted about listening to commercial advertisements on my “non-commercial”, “listener-supported” radio station, as well as the wierd product placement that seems to be sneaking into some segments.
4. Then, last night I gave up on donating to the Tour of Hope. A friend is riding and I wanted to sponsor him, but the online form had some broken Javascript validation that kept popping up windows claiming that my caps lock was on and that I should turn it off. After a couple of those, I gave up on the process.

I think that people think that these things (bad web user experience or quality of guards) don’t really matter. Sometimes I wonder if I’m just being petty. But the thing is, these people are asking for my money and it seems like in return I should expect, at a minimum, that they don’t piss me off. It that really unreasonable?

There’s something odd about the news coverage of Clusty. The New York Times reports that the output is based on results from Overture. However, as far as I can tell, only the clearly marked sponsored results come from us. I’m not sure where the search results come from, but it’s somewhere else.

As usual with Philip, I have mixed reactions to his latest article. I agree with the premise that the typical CS undergrad education in the US doesn’t really prepare people to be successful software engineers. Certainly, I share the opinion that more group projects, and more large, open-ended, and real-world-like projects are needed. I also think that a punch-in-punch-out mentality has to be avoided. (Although unlike Philip, I don’t find that inconsistent with a FortyHourWeek; if you aren’t dedicated and engaged, working longer won’t make you that way.)

I found that this excerpt echoed a lot of my current feelings:

Second, we want our graduates to have satisfying careers where they have autonomy, responsibility, and job security. Consider the typical computer programmer who takes specifications for a subsystem from a manager, unthinkingly produces a program to meet the spec, and hands the resulting code back for testing, integration, and evaluation by other departments. This is really a factory job where the workers' hands don't get dirty.

What's wrong with that? Sociologists have tracked a substantial loss in worker satisfaction as craft jobs were replaced by factory jobs. A craftsman builds a complete product and delivers it directly into the hands of a customer, seeing and hearing the satisfaction that the new product delivered. A factory worker, by contrast, builds a small component of a product and is typically several distribution layers removed from the customer.

A craftsman is responsible for whether and when a product goes out the door to the customer. In a classical factory a totally separate part of the bureaucracy, the quality control department, decides whether or not a product should be shipped. A craftsman listens to customer needs and adapts the resulting product accordingly. Factory workers never talk to customers; a separate marketing department assesses customer needs and determines changes in product design. Not only is a job in a code factory not very satisfying it is neither very secure.

What don’t I agree with? Well, to start with, there is the continuing fear of a brown man. Yes, lots of Indians and Chinese are getting technology jobs. And, good for them! However, I’ve supervised outsourced development groups and I’ve interviewed many developers, and it isn’t at all obvious to me that the Indian or Chinese schools are doing any better than the US ones. The Asians I’ve interviewed in general seem to have retained their ‘book learning’ better than the Americans, but they don’t do substantially better or worse at puzzles or problem solving on average.

I also don’t approve of the idea of removing the traditional liberal arts from the curriculum of aspiring software engineers. We can improve education within the discipline without removing other disciplines. Furthermore, a successful engineer of the sort that Philip would like to see is going to need to be able to communicate effectively, in speech and writing; to read critically and assimilate information from diverse sources; to understand political and macroeconomic trends, as well as the microeconomic forces at play within a business.

Finally, I would add a few suggestions, which might be implicit in Philip’s vision, but which could also be implemented within the current environment. First, students must be required to use version control on any project which involved multiple people, or which spans more than one week. Second, from day one, students should be expected to hand in test suites with every assignment, instead of relying on the test suites provided by TAs.

I was fairly impressed with Kerry’s improvement in delivery in the first debate, but in the end, I’m not sure that it’s enough. His position on Iraq is a tough one and it’s hard to sell. I mean, “how do you ask a man to be the last to die for a mistake?” His Pottery Barn analogy was just a little bizarre, and I think can only be explained by some staffer saying something like: you have to capture the Soccer Moms (oops... according to NPR it’s now Security Moms) vote — toss in something about Pottery Barn or something. Beyond its bizarreness, I will be amazed if it doesn’t get turned around on him: “Kerry says the government broke it, and now he wants your son to buy it.”

Other than that, Bush made a lot more gaffes, although Kerry let him get away with virtually all of them. Some examples:

In response to a question about the importance of Bin Laden vs. Iraq, he slipped and said, “Of course I know that Saddam Hussein is the most important... I mean that Osama bin Laden is the most important...”

Bush would say to troops, “I will bring you home as soon as the mission is done,” apparently forgetting about the mission accomplished stunt. (Kerry made a quick reference to this, but only an hour later. He should have jumped on it immediately.)

Bush looks like quite a bit of a hypocrite going on about the International Criminal Court after the events at Abu Ghraib.

Bush several times insisted on a 1-minute follow-on round and then appeared to have nothing planned to say.

One point of criticism for both candidates is that apparently they both feel that Iran is more important than what’s happening in Darfur, since they both chose to spend the first 30 seconds talking about that country instead of answering the question that was asked.

Sadly, my feeling here, though, is that although Kerry really topped Bush in terms of actually addressing issues that were put forward, unless his campaign can really work the spin engine, the conventional wisdom is going to be that Bush won it because he looked at the camera more and because he appears to have stronger convictions (i.e. is incapable of admitting to any mistake).

when you find yourself wondering, does Ciara really mean what she says in her new single: "Just because you drive a Benz, I’m not going home with you", or did she really intend to say "I’m not going home with you just because you drive a Benz"? Should hip-hop artists be required to understand the difference between dependent and independent clauses?

Today I went and retrieved my books that have been sitting in my old Caltech office for 4+ years. It was a little wierd how little the office has changed. Comics I stuck on the wall are still there, and the desks look more-or-less like they did when I was there. I glanced at the titles of a couple of the papers scattered about and it was all the sort of stuff I would have read. My books were up on the top shelf, but the shelves of the current occupants contained a fairly indistinguishable set of books.

The big thing I forgot when I was figuring how many boxes I would need was my notebooks. I’m not sure what I’m going to do with them, but I now also have several hundred pages of notes from 2.5 years of classes, and research. I did decide to leave the e-prints I had collected in the file cabinet.

I did a quick triage of the books and figured out which ones I’m willing to part with right away. They’re now on my forsale page, on the miniscule chance that anyone reading this is interested in graduate level physics and math books.

A new feature — my Perl Hall of Shame, featuring occasional snippets of perl code that someone actually wrote. Today, a little pattern I’ll call “useless use of $_”:

for ( 0..9 ) {
  my $i = $_;
  for ( 0..9 ) {
    my $j = $_;
    ...
  }
}

I think that perhaps the author of this was trying to avoid polluting the enclosing lexical scope with his loop variables, but wasn’t aware that exactly the same thing can be achieved without resorting to punctuation variables:

for my $i ( 0..9 ) {
  for my $j ( 0..9 ) {
    ...
  }
}

It’s been one of those rollercoaster weeks at work. I really need to figure out how to get my zen on. It’s suddenly become clear to me that the turf wars being fought are much bigger than I realized and that probably no amount of reasoned argument from us down in the trenches will have any impact. Possibly coding like mad will, but I’m not sure; it might just lead to coding like mad in a different direction later. At any rate, it may well be best for my sanity if I just duck my head down for a couple weeks, ignore it all, and get shit done.

This week, I’ve gotten two emails out of the blue from head hunters, both just completely mis-targetted. So, beyond me learning that I should perhaps remove one or two keywords from my resume, I have a couple observations / suggestions for recruiters:

• Where it says “I am not currently looking for work”; I mean it.
• I am not really impressed by your assertion that the job is “very secure” because you’ve had someone working there for 2.5 years.
• 9K concurrent users also does not impress me.
• I’m probably not interested in working on stuff I did in my first year out of college and haven’t touched since.

I have to say that in the middle of a week full of various people suggesting your code is responsible for the company losing scary amounts of money, breathing down your neck for not having fixed the bugs in other people’s code yet, and accusing you of being unprofessional (read “mean”); it is really nice to have someone pull you aside and say “Hey, did you fix bug X in the last launch? Thanks, that was great.”

I picked up Paul Krugman’s The Great Unraveling today. Reading through the introduction, I just about stopped dead at the admonition:

Don’t assume that policy proposals make sense in terms of their stated goals.

I was taken aback by the fact that this was nearly identical to a point that Schneier hammers on in Beyond Fear. For the most part, Schneier is talking about security policy and Krugman is talking about economic policy, although they both make the point that proposed policies in their respective areas may really have more to do with broader goals.

I suppose that in some sense this convergence of ideas isn’t actually too shocking, since both authors actually have fairly similar agendas: to argue that the current administration, aided by the events of September 11th, is making very questionable decisions in order to forward questionable goals, and that this is all fairly obvious if you just think about it clearly.

At Barnes and Noble, they have a display of “banned books”. My first reaction was to note how many of the books on the table I’ve read. My second reaction was to realize just how much more interesting I would have found the display if they had some additional information on why each book was banned, and by whom.

If I ever do get around to taking some classes “for fun”, maybe I’ll have to remember to look for a lit. class on banned books. I imagine there are some interesting stories surrounding the stories.

I just read John Battelle note about Geico’s lawsuits and thought about the supermarket. Most supermarkets hand you a couple of coupons along with your receipt. If you look at them, you quickly realize that most of the time they are coupons for a competing product to one of the items you just purchased. I wonder how this works. Do supermarket advertisers “bid”, as it were, on specific competing products or just on categories? Is this a relevant parallel situation to Google and Yahoo?

(This seems like a good time to point out that I work for Overture/Yahoo, but that nothing I say here in any way represents the opinions of my employers.)

I pretty much keep NPR tuned in the car most of the time, although this week has been tough. In principle I’m actually interested to here what’s going on at the RNC, in a know-your-enemy sort of way. However, I’ve generally found that I actually can’t listen for more than 30 seconds.

While I didn’t catch the whole speech, I got highlights of Arnold this morning and it was completely different. I mean, analytically, he was vastly oversimplifying and ignoring the inconvenient, but it was still excellent rhetoric. This man is dangerous.

I think the whole model year thing has gotten completely out of control. I want to buy a new bicycle, and REI is having a sale, including bikes. However, when I just called to find out what they have and what’s on sale, I found out that they have almost nothing in inventory at the moment, because they are waiting for the new ’05s to arrive.

So, maybe there’s something wrong with me, but I really don’t see why bicycles need model years. I understand that manufacturers may periodically want to change the components on a particular model, but I can’t reasonably figure out why this should need to be synchronized to the calendar.

Okay, it totally sucks that Benchmark only looks at CPU time. Honestly, how often is CPU the bottleneck in a real systems? Granted, it’s pretty simple to write an alternative benchmarking tool for wallclock time, but that’s just all the more reason that it ought to be available as an option in the standard package.

One of our partners, who shall remain nameless, recently reported that the error rate on their queries to us had gone up by a factor of about 10. After a day or two of our tier 2 staff looking into it without identifying the problem, I was pulled in last night. The first thing I asked was, “What do they mean when they say ‘error’?”.

This morning I reviewed what had been done so far with our tier 2 guys, concluded that they had done everything I would have thought of to do and that the problem didn’t appear to be on our end. I then emailed the partner to say that we suspected that the cause was elsewhere, but that we would be surer if they could give us a histogram of the types of errors they were seeing. About an hour later, they came back and told us that it turned out they had accidentally launched some code they shouldn’t have and that the errors were all 404s as they sporadically queried the wrong URL.

Sigh...

There was discussion about Walmart on Marketplace this evening. Apparently someone outside the alternative press finally realized that when companies pay below a living wage and expect their employees to make up the difference by applying for food stamps and the like, in essence the taxpayers are subsidizing the operation of that company. This isn’t exactly rocket science, but I guess it’s news to some.

However, there was a counterpoint from someone arguing that Walmart employs mostly retirees (hmm... employeed retirees, how’s that work?), college students, and the variously disabled, so they’re actually reducing the public burden because otherwise those people would be completely dependent on public assistence. Now, maybe I haven’t seen a representative sample, but that doesn’t match any Walmart I’ve been to, where the employees pretty uniformly appear to be typical working-age, working-class, fully-abled individuals.

In other recent news, I saw a couple newpaper headlines insisting that there was no post-convention bounce for Kerry. However, the Electoral Vote Predictor strongly disagrees. Liberal media indeed...

I think it would be interesting to see a similar graph, but distorted so that size reflected population or electoral votes. Already this graphic is nice in that it reflects a shift from the vast sea of red that we’ve gotten familiar with in recent years, but shrinking some of those wide-open western spaces down to size would really paint a different picture.

Well, looking back, it’s been a pretty good week. Really busy, but interesting and worthwhile. A couple sessions disappointed me, and there were a few things the conference organizers could have done better, but on the whole it was quite valuable.

My big synchonicity of the week revolved around Damian Conway, Paul Graham, and Brian Ingerson (IO::All). I had actually been rereading some of Paul Graham’s essays before the conference, including Programming Bottom-Up, a concept that he touched on briefly in his Tuesday night talk. The basic idea is to extend the language to match your problem domain, thus making it easier and more natural to express your programs. Ingy has created a few examples of this in IO::All and Spiffy. Damian talked about this and more in his Sufficiently Advanced Technologies talk.

At a minimum, this got me to think about some of my basic utility classes and how I could modify them to be more natural to use. But, I’m also thinking about new development and how to keep things cleaner. I need to give some thought, though, to understanding how to ensure that this technology is used for good. If you have a small group of highly talented developers, it is clear that using this power is the way to go. What is less clear is how to scale it to larger development groups. How do you educate people if you change the language itself? And, how do you ensure that this sort of deep magic is used carefully and with restraint?

My other big conclusion was that we can definitely contribute more to this conference. There are interesting problems we are addressing which I didn’t see anyone talking about. To a certain extent, we will be limited by confidentiality, but there should still be general stuff we can talk about that isn’t specific to our application or so critical to maintaining competitive advantage. So, I will probably submit a talk next year, working title “How to serve 1,000,000,000 requests a day with mod-perl”. Keep your eyes open for it.

With one child in college and three to go, the Kent started thinking about the economics of software. Concluded that open source is nonsense. Not saying he won’t do it, or that you shouldn’t, but just that you should be aware that it’s nonsense when you do it.

A goal of society is to apply scarce resources where they do the most good. Money is a tool to do this. Money is information about what is valuable in society (mostly, not perfect). How does the OS movement figure out what is important to do if not with money? Prediction: open source will result is really good software with minimal impact.

Ex., rough estimate of value created by JUnit is $1 billion / year. Kent Beck’s profit: $0. Geek cred is nice, but doesn’t pay for college (or food, or rent, ...).

Maslow’s hierarchy of needs: survival, love, esteem, self-actualization. Must fulfill lower levels before you can proceed to higher levels. But, open source turns this completely on its head. Do what you want, maybe then get recognition in community, maybe then form close relationships, only very few manage to fulfill survival needs through OSS.

Pricing in traditional economic: price set by a seller is a statement of worth. Price paid by a buyer is a statement of perceived value. We (OS developers) think we are making items of value, but to most of the world a price of $0 signals otherwise.

Situations that don’t make sense are warnings of impending change. In what direction will change happen? Hard to say, but if we are aware that change is coming, maybe we can steer it in a direction that is good for the creators of value. But, if you don’t pay attention, MS will capture all the value instead.

Accountability: software development doesn’t have a good history of accountability. Increase in test-driven development is a step towards accountability. OSS tends only to have peer accountability. The problem with your peers is that they tend to let you off the hook. “Oops... it was 3AM and I made a mistake”.

Why does OS seem to work? The first world is a land of plenty. If you have a technical background, you can’t fall that far. At least in the short term, people are willing to sacrifice basic needs for higher needs (doesn’t work so well in the long term, though. Do need sleep, housing, food). Programmers are willing to write software instead of being involved in other hobbies or local community.

How could it work in the future? “Whole people working whole jobs”. Sponsorship (why not USPS JUnit instead of cycling). Patronage (shortage of interested wealthy people, audience member recommends Public Software Fund). Pay-per-use (direct feedback for developers). Licensing. Complementary products.

Preable: big “thank you” to Larry Wall for making this all possible. If everyone in this room got their company to donate $500 to The Perl Foundation, it would be more than enough to keep Larry working on Perl for the whole year.

Desire from someone: implicit $self-> for method calls inside a class module. Sub::CallAsMethod does this. Very simple. Very magical. Probably evil.

This talk is about modules that are indistiguishable from magic.

hmm... blah blah blah... Harry Potter jokes without end...

Modules which do magic without any user interaction aren’t new. Most famously, use strict. More recently, IO::All. But, maybe not magical enough...

use IO::All:Pulp::Fiction
my @horrors = <~/foo.txt>

Lots of stuff in the Perl6:: namespace.

How can mere mortals write S.A.T.? Find the clunky bits of your code and do something about it! IO::Prompt was pretty cool, but originally needed too many flags. Changed to autochomp by default and autodetect boolean context.

Why can’t comments be more useful and interactive? Progress module presented last year weren’t simple enough to use. Solution, smart comments:

use Smart::Comments
while ( ... ) {     ### Preparing--->done
  do_work
}

also provides checks and assertions.

Wrapped up with Lingua::EN::Autoinflect, which is like Lingua::EN::Inflect, but with no interface, just works.

Takeaway: look at the interface of your modules. What could you remove? What could you just infer? Could you have better defaults? Could you use objects with overloaded operators to acheive things with more natural usage?

It’s pretty hard to do this justice, but check out the David Rumsey Map Collection. Amazing historical maps with the ability to compare, overlap, evolve, and knit maps.

Big sites run on Perl: Yahoo / Overture, Amazon / IMDB, TicketMaster / CitySearch. Billions of requests a day served with Perl. How do we do it?

Scaling at this level requires more than just hardware. If your application is slow, you can’t just throw hardware at it. This is about high-level design patterns for applications.

Caching

It’s nearly always worthwhile to cache some data. Caching full pages is best if you can do it. Pregenerate, or generate-on-the-fly and use mod_proxy cache. Usually can’t do this because you have at least some personalized data on every page. Partial page caching caches different page sections with different expirations. Mason provides this. Other modules: Cache::FastMmap (key/value storage in local shared memory), Cache::Memcached (distributed key/value storage)

(Aside: tied BerkeleyDB bad — object interface about 3x faster)

Job Queuing

Bursty traffic is a real pain. If traffic load exceeds steady state capacity, you need to queue requests to be handled in turn. Decouples users from backend load. Typical “search in progress” approach is okay, but doesn’t scale unless you limit the number of worker threads / processes. Don’t want to fork 1000 processes if you have 1000 user waiting. Spread::Queue is one possibility.

Questions...

How do you deal with session data? General rule: keep as little session data as you possibly can.

Memcached vs. MySQL performance? About the same for small numbers of clients. memcached ought to be better for large numbers (at least on BSD or Linux 2.6)

Is Spread ready for production? Seems to be.

Damn... been coding for 5 minutes and I’m wishing I could use IO::All.

Well, I’m figuring I picked the right talk for this time period, since Damian Conway is in the back row as well as a couple other luminaries. If it’s worth his time, it’s probably going to be interesting. (Correction: everyone is here)

Goal: make all i/o as easy as writting to STDOUT. E.g.

"Hello World\n" > io("greetings.txt");

Don’t worry, there are method versions of everything, but overloaded operators seem to make it clearer.

IO::All exports the io() method which is just a factory for IO::All objects. IO::All objects can be files, directories, sockets, pipes, dbms, web pages, and more.

files can slurp(), >>, or scalar ref to get contents.

next() iterates through directories, returning more IO::All objects.

Stuff that just works:

io(‘—’) > io(‘-’);
$tmpio = io(‘?’) < $str;   #tmp file
$tmpio->seek(0,0);
$tmpio > io(‘=’);   #STDERR

<,>,<< on http IO objects does GET, PUT, POST.

2 line forking webserver with CGI support

“IO::All is not your mom”

Continuing on with Spiffy.pm. Sort of a super-exporter that works with inheritance. Also some nice OO conveniences. Implicit $self, can call super to call method of same name with same arguments in superclass. field and const define mutable and immutable object attributes. Provides mixins.

Another standing room only talk, but at least I got a seat. I’m not sure who is doing room assignments, but this seems like it should have been in a larger one. I think there may be a problem of there only being small rooms and really big rooms, and no medium rooms

This talk will cover a subset of best practices used by Ticketmaster to manage a large set of servers with a modest staff. Approx. 800 production linux servers, 4 production clusters, 6000 tickets/minute (max or avg?), very bursty around onsale times. Application is divided into 24 system running on 24 distinct classes of nodes.

Operational culture of TM: automation, open source, reliability, modularity

Best Practices: everything should be active, utilize a flexible system organization, automate everything

Best Practice 1: everything should be active. Use all your hardware resources. Better to automatically decommission known systems, than to automatically commission unknown systems. Idle standby systems often decay leading to surprises when you have to failover. TM uses: multi-master Oracle replication, Active/active VRRP, clustered NAS,

Best Practice 2: utilize a flexible system organization. Operations is often the last to hear about changes. Naming of servers: <class><node#>.<product>.<cluster>.<sa>. All nodes of same class are configured identically; all automation software keys off class. All nodes have two NFS shares: /<class>/local/, /<class>/shared/. Shared typically has code, configuration, templates. Local used for logging.

Best Practice 3: Automate everything.

Configuration management: Kickstart to do OS install. per-class symlink to shared RPM repository. Overlay system copies m4 templates, by class, to root filesystem. All config files are templated. Can deploy a new server in 7 minutes.

Next gen CM (in development): always running to detect configuration drift. TT replaces m4, APT replaces symlinks. Configuration in a giant XML file with all information needed for all classes (is this a good idea?). Configuration authors must also provide auditing functionality. Define severity levels for changes which limit what time windows they can execute during.

Systems Engineering staff is 1 lead, 2 network admins, 3 sysadmins, and 1 developer.

Esther Dyson is stuck in Dallas, so it’s just Freeman and George this morning.

Freeman is mostly interested in biotech currently. He has hopes that this technology with become as common as computers, giving home gardeners, pet breeders, etc. the ability to use it in their work and hobbies.

Tim asks about possible risks

Freeman wonders can it be stopped, should it be stopped, and if so, how do you set up the rules so the it can be used for good purposes and not dangerous ones.

Tim asks George about his experiences leaving the traditional education system.

George comments that he thinks there is a false dichotomy between doing work with you hands and with your mind. On the contrary, good mind work often happens while your hands are occupied with other things. He misses the days when people learned about technology by taking apart carburators and the like. Now, shop classes use CAD instead of actually taking apart and building stuff.

Back to Freeman, Tim references the debates about nuclear technology and asks about how he views society and risk.

Freeman says people are really bad at assessing risk quantitatively. Our society is both extremely risk-adverse and at the same time highly risk-taking, because people don’t think about things correctly.

Tim quoting Mitch Kapor: “Don’t need a Department of Homeland Security, need a Department of Homeland Arithmetic”.

Yesterday was fairly good overall. I enjoyed most of the sessions I went to, and I didn’t overload myself. After the LJ talk, I ran into Geoff Young and chatted with him for a bit about an issue I’m trying to deal with. Unfortunately, I ended up rather late for the Parallel Sessions in Perl talk as a result. He was on slide 11 of 20 when I got there.

Aside: I almost ignored that talk entirely when I was looking over the schedule. That’s because at every other conference I’ve been to “Parallel Sessions in Foo” means something completely different.

After the sessions, I went to off to dinner with a bunch of veggies. We attempted to go to the Calendula Cafe, but they were closed unexpectedly. Instead we went across the street to Vege Thai. I was a bit worried for a bit when the vegans at the table started worrying that the soy “meats” might not be vegan, and I’m just thinking, my god, we went all the way across town to a completely vegetarian restaurant and you people still can’t eat anything on the menu. (N.B., I am not a vegetarian, I just live with one.) But, it turns out their fake meats don’t have egg in them, so it was all good.

Post dinner, all the Yahoo folks headed to a party hosted by O’Reilly, where we all talked to each other. Kinda lame. I wandered a little bit and chatted with a few people, but it was loud and I was having trouble focusing. There were a few people I would have liked to talk with, but I wasn’t feeling up to starting conversations.

Overview of LJ’s evolution from 1 server to ~90 currently including load balancing and distributed caching (memcached). 4 million users, 50 million dynamic page views / day.

Evolution:

• 1 shared server
• 1 dedicated server
• one web, one DB
• multiple web, one DB
• replicated DB
• more webs, more DB slaves, chaos, I/O, CPU contention

Scaling problem: every DB has to write, so eventually you lose no matter how many slaves you have.

Solution: user clusters. All user data stored in one cluster. Never join from multiple users. One global master keeps user metadata (user-cluster map)

But... still lots of points of failure. Add master-master configuration, 2 machines per cluster. Challenges: avoid corruption, unique id collisions, backup has a cold cache.

Custom-built load balancer. Using persistent connections, which helps with burstiness, guarantees connections are good. Paid users get to cut in line.

Replaced RAID with MogileFS. Lost the thread here for a bit...

Caching is important. Would be nice if it could all be handled by the DB, but there are limitations: 32 bit, application logic. Where else to cache? Mod_perl — too much memory use, bad hit rate. Shared memory — tricky, only works on one machine.

memcached is their Open Source, distributed caching system. APIs for many languages. Various machines have different free mem, provide some number of buckets. Hash request, pick a bucket. written in C, event-drived, single-threaded using libevent. custom memory allocator.

10 hosts, 28 instances, 30GB cached, 93% hit rate.

This session ended up standing room only, with me as one of the standing, so I couldn’t take notes at the time. This will be more recollections and commentary

This talk turned out to be about something a little different than I expected from the abstract. In particular, his definition of “enterprise” seemed to include only internal applications that don’t directly make money for the company (but where you can definitely lose money if it doesn’t work). I would have tended to include things like we do: high-performance, high-availibility web services clearly coupled to revenue.

Consequently, I found a number of things I disagreed with. For example, I like metadata a lot and it makes my life a lot easier. However, he seemed to advocate against config files and the like, in favor of keeping all the constants, configuration, and logic in your code.

I did agree with some things. Write tests. Keep methods clear and simple. Use OO (although, I’m not sure I’d go quite as far as he did).

A big area that I expected to be touched on when talking about “enterprise” systems was deployment and monitoring, but that wasn’t touched on at all.

Not there yet, but getting closer. Most of the design, including OO, is complete.

Pipelining operators <== and ==> make map grep max ... a little easier to read and understand. They’re lazy too!

‘fat comma’ makes pairs. Useful for named subroutine parameters, but there is also a new syntax: format $text, :indent[8] :ignore{ /\t/ }

slurp replaces local $/ nonsense

Enhanced sorting removes need to Schwarzian Transform and similar manuevers.

Real classes:

class DogTag {
  has Str  $.name;
  has Int  $.serial;
  has Str  @.record is rw;
  has Base $:posting is rw;
  ...
}

Accessor methods are created automatically. Attributes are read-only unless marked read-write. Private variables marked with ‘:’. Lots of implicit method call stuff, which I’m a little nervous about, but maybe it really is easier to read.

There is a concept of submethods which override a method, but only for that exact subclass, not for sub-subclasses. Supposedly this is good for initialization, but I didn’t quite follow the argument.

Can add “roles” to classes, which are sort of like macros that insert some attributes directly into the class. Good for component-based design. Can create a role without implementations to create interfaces.

The OSCON tutorials are 3 hours total, split in half with a 1/2 hour break. While the break is definitely good, 90 minutes is still too long to go continuously. I think it’s well known that it’s basically impossible for people to focus for more than 45-50 minutes or so. It might have been a better idea to do the tutorials in three 1-hour pieces, or maybe shorten them to two hours total.

Retrospective: This ended up being a little slow, and I suffered from trying to stay attentive the whole time. But, I think it was valuable and I’ll have a much better idea how to approach the docs when I get home and want to play around with this. I’m still not sure if it’ll be something I want to use, mostly because I don’t know what the performance characteristics of the system are.

POE is the core of a system originally designed for a multi-user game with a runtime that could be modified on-the-fly. At the core, basically an event loop that waits for events and calls handlers, which create more events, and so on. Events are queued when the system can’t handle everything in real-time.

Events, Sessions, Kernels

POE events are just lists of data from various sources.

• kernel
• event name
• sending session
• receiving session
• private data location for session
• for I/O events, socket or filehandle
• other application specific data

E.g., a session calls select_read for a socket and alarm_set, which creates events which are enqueued. When an event reaches the head, a callback is invoked (read from the socket, maybe) and this might post a new event (write to the socket). And around and around it goes.

Sessions are somewhat analogous to threads. They have a private data area which is reasonably encapsulated (but this is Perl, so if you really want to break the rules you can).

Sessions can be like “beans” or message-based interfaces. Events invoke a method on an object encapsulated by the Session. POE::Component::SubWrapper does all this for you.

You can use Session to modify event dispatch. Examples: NFA, MessageBased, MultiDispatch

Event fields are members of @_, not a hashref because that’s a lot slower. There are constants exported for field positions. Get your particular stuff with my @stuff = @_[ARG0..$#_]

Sessions only run as long as they have work to do. If they have no events or event watchers active, they get terminated. (Exception: a signal event watcher doesn’t keep a session alive by default.)

Use aliases to allow sessions to send events to other sessions. (Just friendly names, rather than unique numeric identifiers.)

Sessions can create child sessions. A child counts as work for the parent. POE::Kernel is the default child. Sessions get notification if their parent changes or if a child starts or terminates. This is not related to SIGCHLD.

Timer are events set to dispatch at a time in the future. Either absolute time or relative time

I/O watchers work like selects. Can start, stop, pause, and resume. They don’t actually perform I/O, they just watch for readiness. Your handler method should do all that.

Signal watchers look for OS signals or internal signals. Signals sent to a session are also sent to its children. If one session handles it, it’s handled for the whole tree. Makes it easier to do graceful shutdown and the like.

Wheels

Wheels are used to encapsulate common watcher patterns. They are not kept by the kernel, so your application needs to keep a reference. They create and destroy watchers and otherwise modify your session.

Filters translate data for a wheel - serialization and deserialization. Drivers perform file I/O, but basically POE::Driver::SysRW does everything you want.

Output is typically buffered in the driver.

POE::Wheel::Run runs programs and handles their I/O. Can modify permissions and priorities.

SocketFactory makes sockets with callback mechanism for success and failure via events.

Can set high- and low-water marks to detect slow connections and register events to deal with the situation. Also FlushEvent can be set to run when the buffer empies.

Miscellany

Can switch filters during a run. Useful for sending HTTP streams, or parsing headers line-by-line, then slurping up all the body.

Can switch events generated by a wheel. E.g., handle authentication, then pass off to main application.

With one and two halves tutorials under my belt, I’ve got mixed reviews. Damian’s talk was good, the afternoon ones not as good. I’m still trying to get into the mingling thing. There’s a few people I recognize from LA.pm and elsewhere who I haven’t had a chance to go say hi to. However, I did somehow end up having dinner with a big group of strangers last night.

Based on a shortage of sleep, and a review of the abstracts, I decided to skip out on morning sessions today. I was signed up for Patterns in Perl, but the handouts didn’t convince me I would get anything new out of it. I looked through the rest of the abstracts, but wasn’t real psyched about anything. So, I’m going to sit in the lobby and hack a bit and maybe even talk to some of the other people skipping.

Decided to crash a different tutorial after the break. I’m checking out the Subversion session. Sadly, the wireless is not as good in this room.

Subversion has no connection method that supports something like SSH agents. Either cache username and password, or enter it every time.

Copies are cheap, constant-time operations with minimal additional storage required. Makes branching and tagging cheap too (they are just directories in the repository). Easy to change your mind about repository layout.

Hmm... been here 30 min and we’re still essentially on that point.

svn mkdir http://... can create new directories in the repository directly. Similarly for delete. This is handy for working with branches.

status defaults to local changes. Optional to compare to repository.

cvs annotate renamed to svn blame, but also available as svn praise

svn revert to unmodified version (not necessarily the head version). Requires a target and defaults to non-recursive for safety.

Can set properties on files and directories.

merging is always complex, but Subversion is easier than CVS. svn merge -r 15:16 file2 file6 merges the changes in file2 from change 15 to 16 into file6.

Can’t put an SVN repository on a shared disk if accessing directly. In general, using a local-disk repository has issues (permissions, groups). Maybe easier to just set up Apache.

hot backups possible, thanks to Berkeley DB backend.

Speed

DBI is designed for speed, don’t worry about overhead from the frameword. As for your application, pick the right DB, design the schema right. In terms of division of work between application and DB: keep it close to the data, moving data from DB to application over network is expensive. Do more in SQL and stored procedures. But... sometimes Perl is faster than joining in the DB.

Preparing your statements speeds up execution, but can use a lot of time up-front. You need to understand how your optimizer works to get the best performance. Use hints as a last resort, best to let your optimizer do the job, since your key distribution may (will) change.

Remember the SQL cache. Use placeholders to get better cache hit ratio. But, sometimes the placeholder means you can’t fully optimize.

Selecting fewer columns is faster. Using fetchrow_arrayref improves this a lot. fetchall_arrayref is much faster for small number of columns, slower for more.

binding return columns is faster than dereferencing array.

RaiseError is faster than checking return values.

performance profiling in DBI:

> DBI_PROFILE=1 test.pl    # summary
> DBI_PROFILE=2 test.pl    # breakdown per statement
> DBI_PROFILE=6 test.pl    # breakdown per statement and method

DBI Features and Usage

connect_cached saves database handles. Good for oracle, where starting connections is slow. Safer than connecting and reusing because it handles things smoothly if the database crashes or the network goes down or whatever.

Error handling

Errors happen. Failures happen when you don’t handle errors.

RaiseError will call die for you with informative message. You can eval and examine the results in various ways. Or, you can set an error handler with $h->{HandleError} = sub {...}.

From “The Importance of Being Earnest”: Behaving well and feeling well are typically mutually exclusive. Intuitive coding is dangerous. You should be conscious of coding and only depart from guidelines after consideration.

These are D.C.’s guidelines, which might not be best for you. Most important to pick some rules and stick to them.

Highlights:

verb_noun_preposition, verb_noun_participle for subroutine names allows natural language usage. Ex., build_profile_using().

5.6 and later provide aliases from values. Lexical::Alias lets you do some additional tricks.

Implicit localization in for loops - yikes! You can’t break out of a loop and keep the value of the loop variable in the enclosing scope, because the loops is actually using a localized version of the variable which is hiding the version you thought you were using. Probably you should use a subroutine if you need this

List::Util are fast (written in C), you should use them.

You are a developer and a maintainer. So document your code.

There are lots of things that can kill you in Australia; you should go to YAPC::AU.

Don’t use prototypes. They can create non-standard semantics and they don’t work with OO.

Never return undef. You meant return;. (Returns empty list in list context.)

Don’t use bareword filehandles, use lexical filehandles instead. Then put them in braces: print {$file} $num, $name, "\n"

IO::Prompt (coming soon to a CPAN near you) provides nice features for handling user input.

Regexes:

• Always use the /x flag
• Always use the /m flag (make ‘^$’ the same as every other utility, including Perl 6)
• Use \A and \[Zz]
• Always use the /s flag
• Regexp::Auto can do this for you
• Don’t capture unless you need to
• Don’t use $1, etc. Assign immediately, or match-and-assign
• build up complicated regexes from simple units
• Can greatly speed up alternation by factoring out affixes and by using (?>...) (non-backtracking) if appropriate

Readonly, better than use constant?

croak, don’t die. Makes it easier for the user to figure out where they screwed up.

Never cut-and-paste cut code except into a subroutine. Never cut-and-paste a subroutine except into a module.

After reading various posts and comments from some of the presenters, I’m feeling some pre-conference jitters of my own.

In particular, I’m a little distressed by an attitude that seems to be moderately common among those speaking, which is something along the lines of “I wouldn’t be here if I weren’t speaking; None of the sessions are anything new or interesting to me”. There are a few possible explanations I can see.

First, the people in question could just be close-minded. I certainly wouldn’t be the first time I’ve run into people who can’t find anything interesting outside of the narrow band of work they do.

Second, maybe the talks really are mostly just rehashes of stuff you can find in the manuals or FAQs. This would be a little sad, and would reflect badly on either the conference organizers or the speakers themselves (for not putting in the effort to talk about anything that isn’t already well documented).

Third, maybe there really just isn’t anything new and interesting to talk about, at least in certain areas. I’ve heard some people make the claim that Perl is getting stagnant and boring. I’m not sure that I really buy that, though. More mature, yes, but that opens up a whole new area of interesting work to do. (I’m hoping that the POE talk, for example, fits into this category.)

Possibly, I’m just overreacting to passing comments. I’m still trying to keep an open mind at this point, although, I’m also taking a little advice from Dr. Bacchus to think about exactly what I want to get out of the convention. I don’t think I’ll go into precisely what that is just yet, though.

We flew into Portland yesterday afternoon and proceeded to hide in the hotel room from the 104 degree heat. Yikes! Around 10PM it finally cooled down enough to venture outside for a trip to the local supermarket. Oddly, we had to call down to the front desk to get a mini-fridge for the room.

The conference wireless isn’t set up yet (not too surprising), so I ordered the in-room internet access for the weekend. I’ve never actually used one of these crazy hotel internet services before. At first, I thought it wasn’t working because you have to open your browser and do some wierdness there first before you can make connections on any other ports. For a moment I was worried that they thought that "the Internet" only included port 80, but after clicking though some usage agreement nonsense, my mail and ssh opened up.

Today, I think we’re exploring the town. Tomorrow, mondo croquet, Powell’s, and maybe the brewer’s festival if the weather is cool enough.

This morning, I took the Programming Concepts test. I was slightly disturbed that I actually scored higher on this one than the Perl test, since I felt significantly less confident in my answers. (Although, I realized late last night that I made at least two stupid mistakes on the Perl test by skimming perldoc a little too fast.)

I also felt like this test wasn’t as good as the Perl one, probably as a side effect of covering a broader and vaguer subject area. In particular, the language used in a number of questions (especially related to OO concepts) was rather confusing and unclear, in my opinion. Given the debates that regularly take place on some of these topics, it seemed like a guessing game of which camp the author of the question was in. There was also at least one debugging question that seemed like a matter of opinion (debugger and breakpoints vs. isolation and unit testing).

We’re contemplating ditching our home-grown test for job applicants and requesting they take one or more Brainbench tests instead. But, of course, we need to evaluate those tests and get some normalization. So, I took an hour this morning and took the Perl test. I am now officially a Perl Master. Whoot.

The test was fairly decent, although I was surprised how few, and how simple, the regular expression questions I got were. There were only about three or four and they were almost complete just about whether you know about maximal vs. minimal matching. I got a lot more questions about symbol tables and globs than I expected. I guess maybe it focuses in on that type of thing if it decides you are pretty good (the test is adaptive), although that’s the sort of thing that I think should only be used with extreme caution, even by experts. I also got one or two weird questions where I’m half-convinced there were typos or mis-wordings in the question. Unfortunately, I didn’t think to note them down at the time.

Addendum: I forgot to mention that there were a number of questions along the lines of, what does this script output, or, why does this fail? The rules for the test say that you can use reference materials, physical or online. They don’t explicitly forbid just running the code, but it seems like that shouldn’t be allowed. I restricted myself to Programming Perl and perldoc, but it was definitely tempting to just run the code a couple of times. This seems like a concern if we really want to use this for assessing candidates. I felt like I ought to email Brainbench and ask for clarification about this.

I recently came across a link for this article: How to Misuse Code Coverage, about how not to use code coverage tools. (Interestingly it was in another person’s meditation on this idiom.) It’s pretty good, and cogently explains the vague feelings I’ve had keeping me away from certain parts of our code where I know I could trivially up the code coverage, but wouldn’t really test the important things (which, sadly, I don’t have time to write tests for at this point).

Apparently Boston’s T is planning to start searching passengers, out of increased worries due to the upcoming Democratic National Convention. The ACLU is protesting. I had a few thoughts on the matter.

• First, when will civil libertarians ever learn that the way to get support is not to say things like, “Well, you know, people do use illegal drugs, and maybe they forget that they have a joint in their backpack, and those people are going to get in trouble if they get searched.”?
• This won’t be universal searches, which would cripple the system. Rather they will be “random” searches, with all the usual concerns that brings.
• I’m not the first to say this, and I won’t be the last, but I’m really sick of hearing people say, “things are different after 9/11”.
• You really can’t say things like, “This is meant to be a deterrent. Nobody believes we’re going to approach 100% with this.” You’ve basically just told would-be wrongdoers, “we probably won’t catch you”. Deterrance only works if the chance of getting caught is high and the penalties are sufficiently severe. You have to figure that a possibly suicidal attacker is extremely tough to deter.
• Does this mean you’re not going to be able to travel on the T with a *gasp* pocketknife anymore?

There are days when I become totally disheartened and convinced that I’m the only person around here who actually tests their code. Somehow it seems that every time I take a nearly-finished project and start work on integrating it into the main code-line, I find that various tests are failing, leaving me to track down why exactly and find the person who made the change to confirm if it is a bug or an evolution that invalidates a test assumption. Since I’ve typically unit-tested my code pretty thoroughly by this point, I nearly always spend significantly more time dealing with other people’s bugs than fixing any problems I find in my code.

Given this, I have to wonder why I bother writing test frameworks, writing convenience wrappers, writing convenience wrappers to the convenience wrappers, writing documentation for the test systems, setting up nightly test runs and reports, giving talks about the test systems, coming to meetings to consult on the use of the test systems, harping on the unit tests (or, more typically, lack thereof) in code reviews, etc.

sigh...

After seeing it show up, with positive words, on enough people’s blogs, I gave in and picked up a copy of Eats, Shoots & Leaves. I’ve only finished the first chapter, but I’m a bit bothered by one of the examples.

Truss repeatedly mentions the movie title, “Two Weeks Notice”, as an example of the decline of the apostrophe. Apparently, she thinks that it ought to be “Two Weeks’ Notice”. Now, the problem here is that, as far as I can tell, the notice does not belong to the two weeks in question. On the contrary, I would consider “two weeks” to be an adjective phrase modifying the notice. Arguably, the plural is inappropriate (compare “ten gallon hat” or “five pound sack”), but I’m finding it hard to argue for an apostrophe being involved in any way.

(Skipping ahead a bit in the book, I find that she has a special rule in her list of uses for the apostrophe, for just this case (indication of time or quantity). I find this uncompelling, though, since universal application of the rule seems nonsensical. Garner seems to agree with me, although I suppose it’s possible that this is a British-ism.)

I’m feeling like maybe I’m being a little slow, but I wish I could figure out what the lesson is that Tim Bray wants us to take away from his recent post about Latency. Is is that context switches are slow, so running a client and server on the same CPU is a bad idea? Is it that Apple’s network layer is slow? Is it that Apple’s JVM is slow? Is it that Apple’s hardware is slow? Is it something about TCP/IP?

I’m leaning towards the first, but I kinda wish he’d just come out and tell us what it’s actually instructive of.

It seems that there’s two basic types of dom