Petter Häggholm’s blog

Do any of you guys know much about Apache and mod_rewrite? I could use some help.

Update: Chutz asked me the rather obvious question, had I tried turning off all the RewriteConds? The rather sad answer is that no, I’d missed that obvious debugging step. When I did, the RewriteRules worked… With a bit of help from a very high log level, it turned out that while mod_rewrite applies the RewriteBase to the URI (here, truncating the directory) when applying the rule, it does not apply the RewriteBase when matching against a RewriteCond. Thus, the solution is to write my rules as below, but to insert the directory name—the same directory name as the RewriteBase!—in the matching rules, e.g. ^/newsite/\w+.

I’m playing around with some stuff (on my local box, so far, though I’ll be replicating it at Webfaction…if I can get the damned thing to work) with a dynamic website that uses mod_rewrite to take extensionless URIs and turn them into script invocations (mod_wsgi, as it happens, moving away from the largely-deprecated mod_python). This works beautifully when I only have one site. Now, however, I want to have two sites in different <Directory> sections in the same <VirtualHost>, and things aren’t working so smoothly. In fact, as soon as I change my DocumentRoot to something other than the path of the <Directory> the RewriteRules seem to stop working, even without adding a second <Directory> section.

All I get for every request in the mod_rewrite log is a notification that it passed through:

(In the Apache error.log, of course, I get the expected error messages about requests for resources that can’t be found.) I’ve tried to add an appropriate RewriteBase, but so far to no avail. My current setup looks like this, and doesn’t work:

Listen 80
LogLevel info
LoadModule wsgi_module /usr/lib64/apache2/modules/mod_wsgi.so

WSGIPythonPath /home/petter/projects/newsite:/home/petter/projects

NameVirtualHost 127.0.0.1:80
<VirtualHost 127.0.0.1:80>
	ServerAdmin webmaster@localhost
	RewriteLog /tmp/rewrite.log
	RewriteLogLevel 2
	
	DocumentRoot /var/www/localhost/htdocs/wsgi
	<Directory "/var/www/localhost/htdocs/wsgi/newsite">
	        Options Indexes FollowSymLinks ExecCGI

		AddHandler wsgi-script .wsgi

		Order allow,deny
		allow from all

		RewriteEngine On
                RewriteBase /newsite

                # Really, really annoying; the trailing slash fixes don't seem
                # to work on the server's document root...
                RewriteCond %{REQUEST_URI} ^$
                RewriteRule ^.*$ test.wsgi?page=index [QSA]

                # Redirect .py files
                RewriteCond %{REQUEST_URI} ^\w+\.py$
                RewriteRule ^(\w+)\.py$ test.wsgi?page=$1 [QSA]

                # Redirect extensionless URLs, unless they're for directories
                RewriteCond %{REQUEST_URI} ^\w+$
                RewriteCond %{REQUEST_FILENAME} !-d
                RewriteRule ^(\w+)$ test.wsgi?page=$1 [QSA]

                <Files *.xml>
                    Order Deny,Allow
                    Deny from All
                </Files>
	</Directory>
</VirtualHost>

// 1.
$map[$value] = ($value == $req['value']) ? 1.0 : 0.0;

// 2.
$map[$value] = ($value == $req['value']) ? 1.0 : 0;

Can anyone think of any reason whatsoever why these two statements should behave differently? If you had told me they would, I would have laughed derisively. And yet, PHP 5.2.6† at least thinks that they are not merely different, but critically so: While (2) works, (1) results in a syntax error:

Parse error: syntax error, unexpected T_DNUMBER in [...].php on line 232

Note that

1. the literal 0.0 is not illegal in general, and
2. the statement fails with other floating-point literals, too—it may be irrelevant to write 0.0 rather than 0, but I also couldn’t write 0.5 if that were what I needed.

What the hell is this lunacy‽

Update: This must be a bug, not (another) idiotic design feature: It raises a parse error when I run it through Apache/mod_php‡, but not with the CLI version of the PHP interpreter. On the other hand, why on Earth should the two use different parsers…? The mystery only deepens.

† petter@petter-office:~/temp$ php --version
PHP 5.2.6-2ubuntu4 with Suhosin-Patch 0.9.6.2 (cli) (built: Oct 14 2008 20:06:32)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
    with Xdebug v2.0.3, Copyright (c) 2002-2007, by Derick Rethans

‡ I often wonder if it isn’t really mod_intercal. PHP is but a PLEASE and a COME FROM away from being INTERCAL 2.0 (for the Web).

…Particularly when you’re never quite sure whether some function or object expects (or returns) local or UTC time, and when your database uses at least three different formats for storing them. (I officially hate timezones, and will let this be a reminder never to use anything but UTC for storing dates in an application, ever.)

Subject: The time box issue/GM date problem

I *think* I finally fixed it -- please test and verify. (I checked the
configuration tab, which seems to behave properly; I verified that the
profile displays matched, and I verified that saving a time returns the
same time rather than an off-by-an-hour one.)

That was an incredibly irritating bug, and I missed two jiu-jitsu
practices over it, but I introduced it by making a bad assumption, so I
suppose that's fair.

My bad assumption was that I could take a time T, take its timezone
offset, and by adding (hours*3600+minute*60+seconds) and adding the
offset back, I'd get a good time. That was stupid.

The problem is that different timestamps, from different dates, come
back with different timezone offset. I would suspect this may have to do
with policy changes in things like timezones and DST. Specifically, the
timezone offset from UTC on the same machine differs by 3600 seconds --
one hour -- for timestamps in 1970 and timestamps in 2008, respectively.

On the bright side (and the reason why I made those changes in the first
place), we can now say things like

	$d->toTimestamp()

instead of

	date('G', $time_inseconds)*3600
	         +$minutes_default*60
	         +$seconds_default;

which I think is worth it in the long run. Or so I tell myself to
console myself for the two missed jiu-jitsu practices.

-- 
Petter Häggholm
eRezLife Software
http://www.faqs.org/docs/jargon/T/top-post.html

I read an interesting article by Joel Spolsky, In Defense of Not-Invented-Here Syndrome. It’s worth reading, but in case you're feeling lazy, here are the highlights:

When you're working on a really, really good team with great programmers, everybody else's code, frankly, is bug-infested garbage, and nobody else knows how to ship on time. When you're a cordon bleu chef and you need fresh lavender, you grow it yourself instead of buying it in the farmers' market, because sometimes they don't have fresh lavender or they have old lavender which they pass off as fresh.

[…]

The best advice I can offer:

If it's a core business function -- do it yourself, no matter what.

Pick your core business competencies and goals, and do those in house. If you're a software company, writing excellent code is how you're going to succeed. Go ahead and outsource the company cafeteria and the CD-ROM duplication. If you're a pharmaceutical company, write software for drug research, but don't write your own accounting package. If you're a web accounting service, write your own accounting package, but don't try to create your own magazine ads. If you have customers, never outsource customer service.

Now, I absolutely agree with his best advice—if it’s a core business function, do it yourself. After all, it’s your business to perform core business functions well, and if you can’t (or don’t) do it better than your competitors, you don’t have a selling point. But to imply even loosely (he does take the edge off it slightly later in his article) that because you write software, all of software is your core business function, is, I think, so hyperbolic as to be completely misleading. The worst part of the article follows:

The only exception to this rule, I suspect, is if your own people are more incompetent than everyone else, so whenever you try to do anything in house, it's botched up. Yes, there are plenty of places like this. If you're in one of them, I can't help you.

How about an exception for when your people have better things to do? We use all kinds of libraries, internal and external. We use an external DB compatibility layer, various internal tag generation and Javascript DOM libraries, an internal ORM, an external email library with internal wrappers… According to the Spolsky logic, here, we should stick with the internal Javascript library, because our devs know our needs best (true); we should stick with the internal ORM; we should possibly even contemplate our own internal DB layer…this has in fact been brought up, and I think it’s a terrible idea.

Because, let’s face it, DB layers and fancy DOM manipulations are not our core business functions. Our core business functions are things like writing very flexible management systems for medical schools and university residences with client-defined workflows and customiseable forms and reports. This is what we do. This is what we sell. This is what our clients pay us to do, and what they cannot get from anyone else (at competitive prices).

This is why I think we should phase out our internal DOM stuff in favour of jQuery (which we recently adopted), why we should stick with MDB2 for a DB layer, and why I'd be happier if we had a third-party ORM. Is it because I think that the MDB2 people, the jQuery people, and the hypothetical ORM people are smarter than us, or better programmers? Not at all. It’s because that’s not what we’re paid to do, and unless the tools are really bad, it’s going to be vastly more productive to work with the third-party tools, find other third-party tools, or help fix the current tools, in some order of preference.

Maybe Microsoft can afford a team to make a custom compiler for Excel. We sure as hell can’t. We’ve got enough bugs and feature requests to worry about as it is, and almost none of them are ultimately the fault of any third-party library. (I’ve made three bug reports to the MDB2 project, I believe—I can assure you that we have rather more bugs than that in our tracker.)

Now, I’m sure that Joel Spolsky is a smart guy who knows all this, and may have known it before I first used a keyboard (there’s a reason I don’t send this as an email to Joel Spolsky). However, given all these ifs, buts, and general caveats, what’s the point of that article? If the real gist of the article, put succinctly and honestly, should be Not-Invented-Here Syndrome is not a valid concern for your company’s core competency, nor if adequate third-party solutions cannot be found or afforded, I think this is a job for…

I bet you have! I bet, furthermore, that you've wondered how to do it gracefully, especially with XHTML, which doesn't allow document.write() which is the only Javascript-based solution that LiveJournal offers out of the box.

Why do I want to use Javascript? Well, I could use a server-side script to extract the posts, but that's precisely the problem—it'll be serverside, and why should I make it so? Ultimately, the transaction is between the client and the LiveJournal server. There's no logical need for my webserver to fetch the contents, then paste and send the contents of the blog when the client is perfectly able to fetch it himself. Other solutions, like <iframe>s, are little less unsatisfying: They're ugly and break the page layout.

But how can I gracefully embed the blog in a webpage without document.write()? I thought for a while of various horrible schemes—could I, for instance, create a hidden <iframe> and extract its contents on page load? I'm still not sure if it's possible; it would certainly be cumbersome, and if it is possible it seems to expose the user to cross-site scripting attacks. (If it is possible, and if my <iframe> source referred to anyone else's LiveJournal, I could Ajax it right back to my server, hidden posts and all—because frame solutions use the client's cookies to determine privelege levels.)

The answer is here, it uses JSON, and it is customiseable, although it takes a wee bit of tweaking and (unfortunately) some poking at S2 layers. The credit belongs with slothman.

One of these days I'll actually stop tweaking the mod_python-based version of my website and launch it…

People who actually understand threads are rare and strange and twitch a lot. They tend to wake in a cold sweat in the middle of the night and start raving about race conditions.

Paul Harrison, in a Daily WTF comment

class someClass
{
    /**
     * This is the class constructor.
     */
    public function __construct()
    {
        // ...
    }

    /**
     * This method takes two parameters, an integer
     * $my_id and a string $some_name.
     *
     * @param int $my_id My ID.
     * @param string $some_name A name to save.
     */
    public function foo($my_id, $some_name)
    {
        // ...
    }
}

I'm glad the comments are there to tell me these things.

At work, as previously mentioned, I'm a great champion of phpDocumentor, and I take some pride of being the moving (read: nagging) force that resulted in our now having a repository of generated documentation for the core parts of our system. Unfortunately, there's a bug that prevents us from running it on the entire codebase, so we have to run it in pieces on the really interesting bits—run it on our whole codebase and it'll eat up all memory and die horribly. (It even dies on my desktop, which has a respectable 4 GiB of RAM. It really shouldn't. Our codebase isn't huge.)

One solution that has been proposed (as seen on the previously referenced bug page) is to move the data handling part of the documentor to a[n] SQLite database instead of keeping it all in memory. Apparently, PHP's string handling is rather inefficient, and there are a lot of cross references—this is why we use a parsing documentor rather than something that merely reads docblock headers, after all!

So I figure, what the hell, I have some free time to kill, I'm an open source geek, and improving this would make work life easier. At the same time, I can't afford to do this at work—or rather, my company can't afford to spend developer time on external tools rather than our own software—so I figured I might spend some evenings taking a look at this and seeing if I can make any headway.

A quick look at the codebase tells me that…well, the phpDocumentor code references all the memory bits through arrays rather than method calls, and it makes use of a lot of public class members. In fact, if I make all the member variables of one of the classes I suspect would need some surgery protected, the program ceases to work. There are many classes, many with a good ten or twenty public variables, promiscuously referencing the public members of other classes (ohoho), and no data encapsulation API whatsoever to hijack.

I'd like to see this improve. I'm presently a bit uncertain of whether I'd really like to spend my free time on it…

This is brilliant.

HTTP 1.1, as per RFC 2616, supports no less than three ways of specifying the length of the response to a simple GET request:

• In a Content-Length header, as in HTTP 1.0;
• By means of the header Connection: close, which means content follows and ends only when we close the connection on you;
• By means of the header Transfer-Encoding: chunked, which means that the message is transferred in chunks, each of which begins with a chunk size—in hex, unlike the decimal representation of Content-Length—and ends with good old CRLFCRLF.