Change? What change? And screw the DNC while we’re at it…

I’m obviously biased, but I’m beginning to become quite infuriated that so many people are gobbling up Obama’s ridiculous notion of bringing change to Washington.  Seriously folks, what is this man with no substantive policies going to do in DC?  Hell, with all his “present” votes, he doesn’t even have substantive votes.

I wish people would seriously consider what is going on in the world instead of blindly embracing some free-love, dreamy notion that everything is going to be all better.  Obama will kiss all our boo-boos goodbye and we will scamper off to the playground for another scrape and fall.

This is simply maddening.

And the whole Florida debacle, yet again, although in this case they’re the victim and not the culprit.  The DNC has overstepped its grounds in its baseless, anachronistic punishment of FL voters by depriving them of their delegates at the convention.  A message to Howard Dean:  I like you and all, seriously, but get a fucking grip.  The times of slowly coronating nominees is over.  It’s time for a national primary and it’s time to realize that the red-necks in Iowa aren’t really “notoriously finicky and pensive about politics” but rather that they more accurately sport a convoluted and backwards caucus.  Screw Iowa.  There’s no reason why any of the four states “allowed” to break party rules and hold a primary before Super Tuesday.  It is completely arbitrary.

Think about it… people want all this change.  What if we did have a national primary?  What if instead of people constantly being worried about electability, there was one feel swoop of voters voting and making a decision.  A simple plurality of votes would do it.  Instead, we ween “second-tier” candidates out of the process because they’re not “electable.”

I reject the notion that these four states provide a temperament of the country.  They provide a temperament of their states, nothing more.  It’s time for a national primary and in 2008 it’s time for a candidate that really will be able to change things from day one:  Hillary Clinton.

Obama’s Reagan Comparison

Obama stepped in some doo-doo with his remarks about Reagan recently while interviewing with the Reno Gazette-Journal. I’m going to snatch the quote from this Washington Post blog post:

I don’t want to present myself as some sort of singular figure. I think part of what’s different are the times…I think Ronald Reagan changed the trajectory of America in a way that Richard Nixon did not and in a way that Bill Clinton did not. He put us on a fundamentally different path because the country was ready for it. I think they felt like with all the excesses of the 1960s and 1970s and government had grown and grown but there wasn’t much sense of accountability in terms of how it was operating. I think people, he just tapped into what people were already feeling, which was we want clarity we want optimism, we want a return to that sense of dynamism and entrepreneurship that had been missing.

Or if one desires to see a talking head, Obama’s remarks can also be found on YouTube.

Now it’s time for the fun part.

Wait a second. How much was the federal deficit when Reagan came into office? $900 billion. And when he left? $2.7 trillion. Check this site out for some interesting statistics on the Sad Legacy of Ronald Reagan. At yet, in a mind-boggling fashion, people still place the “mantle of fiscal responsibility” with the Republicans. And now we have take two with the dunce in the White House who has repeatedly requested raising of the federal debt ceiling. According to that 2004 piece in the New York Times:

Since Mr. Bush took office in January 2001, the federal debt has increased about 40 percent, or $2.1 trillion, to $7.4 trillion. Congress has raised the debt ceiling three times in three years, raising it most recently by $984 billion in May 2003.

Mr. Obama, I can only surmise from your comparison with Reagan that you too intend to spend us into oblivion like Reagan and George W. Bush. I can only surmise that you plan to destroy our environmental policies, oppress unionized workers, and push for tax benefits to the super-rich.  That’s the kind of change Reagan and George W. Bush have brought about. Sorry, count me out.  I don’t think he’d do those things, but by invoking Reagan’s name solely to pander to right-leaning folks, he is deserving of the baggage that will come along with it.

Obama has so very little substance it’s kind of unnerving to me that he is driving so much support. The Reno Gazette-Journal Oped piece goes on to suggest that Obama should be the party’s nominee even after they say:

One can fairly describe Obama’s philosophical optimism and charismatic manner as too idealistic, even a tad dreamy.

Umm. The last thing we need is another foolhardy person in the White House with a free-love mantra. Come on, folks, how long will such a thing last after January 20th, 2009? A week, maybe two? One should not confuse hope of change with visions of grandeur. The fact of the matter is that to get things done in the government, you have to know how to work with the people in government. Bill Clinton showed remarkable gains across the aisle because ultimately he was a centrist. The Republicans hated this fact and still hate it because Hillary Clinton is a centrist as well. Why do they hate it? Because it beats them at their own game of fearmongering and wedge issues.

We desperately need someone in the White House who, from day one, can begin to make inroads domestically and internationally to reconstruct what George W Bush has destroyed over the past 7 years. In the words of the Reno Gazette-Journal:

Hillary Clinton should be the party’s choice.

Edit: I just realized that today is January 20th which I find amusingly appropriate for such a post. Unfortunately, 2008 is a leap year so we have 366 days until The Doofus is out of the White House instead of 365. But still… only 1 year left!!

postfix/qmgr warning: connect to transport spamassassin: Connection refused

While setting up a postfix/spamassassin/spamc combo, I got everything up and running but was being harried by log messages like so:

postfix/qmgr[547]: warning: connect to transport spamassassin: Connection refused

It turns out that this message isn’t necessarily indicative of continued problems of messages being delivered through postfix and spamassassin, but rather of messages in the queue when the setup was screwy. I found this post about bad mail stuck in qmgr with a one-line reply with the clue that pointed me in the right direction.

During the course of the setup, I fat-fingered some stuff, made some minor mistakes etc. On each iteration, of course, I was sending myself test emails. So again, the log message noted above was showing up in my mail.log. When I got everything working, it persisted.

The cause? Mail in the queue sent previously that was still trying to do Bad Things. In my case, this manifested itself like so on a run of `sudo postqueue -p`:

AFEDB98077      495 Sat Jan 19 17:56:56  rwoodrum@domain.net
                                                  (mail transport unavailable)
                                         rwoodrum@domain.com

The solution? Delete those messages from the queue (or requeue them) with postsuper and restart postfix. Log messages gone, mail being delivered through spamc/spamd combo. Now, of course, this is only to get rid of the log message after you’ve got everything setup correctly. If your config is still borked, the message is obviously going to persist. ^_^

Mike Huckabee isn’t radical … for a bigot, anyway

While it should come as no surprise for anyone that knows anything about him, CNN’s political ticker had a nice little snippet on Mike Huckabee’s gay marriage comments:

In an interview with the religious Web site beliefnet.com, Huckabee pushes back on recent critics who have called some of his positions “radical.”

“I think the radical view is to say that we’re going to change the definition of marriage so that it can mean two men, two women, a man and three women, a man and a child, a man and animal,” he said in the interview, published on the Web site Wednesday. “Again, once we change the definition, the door is open to change it again.”

So you can see he’s really not all that radical - for a bigot, anyway.

“The door is open to change it again.” Interesting he should say such a thing. You know, again, he’s right… because first we did dastardly things like the Fifteenth Amendment to the Constitution where “The right of citizens of the United States to vote shall not be denied or abridged by the United States or by any State on account of race, color, or previous condition of servitude.” And it was all downhill from there. Next thing you know women could vote with the ratification of the Nineteenth Amendment to the Constitution.  We’ve never recovered from these two shenanigans!

Leave it to the Republicans and the fanatical religious right to want to use the Constitution to deny rights rather than to grant them.

ruby 1.8.5 not handling SIGTERM

I’ve deployed some applications using the nifty runit project because it offers very nice mechanisms for process supervision. I’ve been using it flawlessly in several environments for at least a year… until just the other day when I was sure I had found a problem with it. My investigation of sv term handling with a slow child process is available on the runit mailing list.

The problem I was experiencing was with a ruby application running under runit and not honoring SIGTERMs. Looking at the 1.8.0 runit source via `apt-get source runit` on debian sid, I learned more about how runit’s pieces work together to manage and supervise child processes. Riddling the code with debug info, I was able to trace the offending code to the sv.c source file in the control() function. A snippet of code was disallowing writing of another command to a pipe for later processing because the process was already in a particular state. In my case, runit thought the process was already down because I had previously sent it a TERM signal. Unfortunately, my process never handled the original TERM because of delayed registration of the handler and therefore never would receive another from runit.

This repro’d 100% of the time on ruby 1.8.5 with this snippet of code:

puts "doing initial sleep for 10..."
sleep(10)

puts "registering term handler..."
trap("TERM") do
  puts "got term"
  exit
end

while(true) do
  puts "in loop, sleeping for 2..."
  sleep 2
end

Here’s where it gets a little messy…

As I outline in my post to the mailing list, runit does indeed refrain from sending a TERM signal again to the process, but the real offender was ruby v1.8.5. The Google led me to a post on ruby-forum about a signal handling behavior change on an upgrade to ruby v1.8.5. Matsumoto chimes in and indicates that this seemingly related(?) issue is indeed a bug.

I tried a most base case running a ruby script consisting of ’sleep 30′. Running this under:
rwoodrum@fs1sea:~/tmp$ ruby -v
ruby 1.8.5 (2006-08-25) [i486-linux]

and subsequently sending the process a TERM does nothing. A la this annotated strace:
..... snip .....
..... we received a TERM .....
) = ? ERESTARTNOHAND (To be restarted)
--- SIGTERM (Terminated) @ 0 (0) ---
sigreturn() = ? (mask now [])
select(0, NULL, NULL, NULL, {12, 476000}

..... riding out the sleep .....

) = 0 (Timeout)
time(NULL) = 1200560526
sigprocmask(SIG_BLOCK, NULL, []) = 0
sigprocmask(SIG_BLOCK, NULL, []) = 0
rt_sigaction(SIGINT, {SIG_DFL}, {0xb7f180d0, [], 0}, 8) = 0
exit_group(0) = ?

I believe at this point that the SIGTERM handler is going to be SIG_DFL. I don’t understand why this doesn’t perk up and terminate the process however, so that is still somewhat unsolved. This doesn’t repro with a more recent version of ruby.

Lessons learned? The version of Ruby1.8 in debian etch (package ruby1.8 version 1.8.5-4etch1) does not appear to handle SIGTERM correctly, even in cases where a signal handler is registered.

2008 Presidential Candidate Lawyer Profiles

Warning: Shameless plug incoming.

As is the case with many politicians, of the 2008 presidential candidates that are considered “frontrunners” according to the Washington Post 2008 Presidential Candidates profiles, it may be no surprise that several of them are lawyers. It’d be interesting if more of them had information in their lawyer profiles. I would be very curious to see what sort of information their campaigns/spokespeople/whatever included. Avvo doesn’t cover every single state yet, but there are some profiles of top-tier (and not-so-top-tier) candidates:

Maybe there should even be a special section for our esteemed (and not-so-esteemed) POTUS’…

I wonder if they’d answer policy questions in the Avvo Answers section… hmmm….

Oh… and just in case anyone cares… the views on this blog aren’t reflective of the views of my employer etc. etc.

Thank you, NH.

Hillary for President. I know that’s not much of a substantial update, but I had on my todo list today a blog post. It’s heartfelt nonetheless. Hillary Clinton is the best candidate to unsling America from the mud that George W. Bush has managed to dredge us through so well.

George Bush, you ought to be ashamed of yourself. You are a terrible leader and do not have this country’s interest at heart. You have done a disservice to this country and are non-deserving of the office you have held. As a human you are pitiful, as a president, you are criminal.

submit a sortable_element in a form

Today I needed to use some of Scriptaculous’ nifty sortable_elements inside of a larger form for a rails app I’m working on. The way I did it works, but it is doubtlessly a terrible hack. Whatever.

Being a complete javascript newb, I took my cue from this post on the rails mailing list which unfortunately, as the guy requested, did not have the answer bundled up nice and neatly for me. On the other hand, it was pretty easy to do by updating a hidden field with the sortable_elements to get the data submitted with the form. Onward with an example…

In my particular case, I have two lists which I drag and drop between with sort order being important in the destination list. These sortable elements are part of a form which the user will post to the app. In my form rhtml template, I have a submit_tag like so:
<%= submit_tag("Add", :onclick => 'sortableToHidden()') %>

The layout for the template in question has a small snippet of javascript:

function sortableToHidden() {
  document.my_form_name.sortable_hidden.value = \\
    Sortable.sequence('my_sortable_list_id');
}

Where:

  • ‘my_form_name’ is the name of the form within which I want to submit the data
  • ’sortable_hidden’ is the name the hidden field in the form
  • ‘my_sortable_list_id’ is the id given to my sortable list in the sortable_element helper

That’s pretty much it. You can then easily access your sorted list via params[:sortable_hidden].

I’m sure there’s some much more nifty, magical way of doing this (which I would love to hear about) but this gets the job done.

Apache2 - error running shared postrotate script

So something happened not too long ago such that in one of our Xen development environments (running Debian etch), apache was bailing every Sunday morning at a conspicuously consistent time.

This fairly obviously pointed to something going on inside of cron with the likely suspect being log rotation. Even more of a no-brainer given cron’s weekly complaints:
/etc/cron.daily/logrotate:
apache2: Could not reliably determine the server's fully qualified domain name, using 10.1.4.24 for ServerName
apache2: Could not reliably determine the server's fully qualified domain name, using 10.1.4.24 for ServerName
(98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
error: error running shared postrotate script for /var/log/apache2/*.log
run-parts: /etc/cron.daily/logrotate exited with return code 1
So a bit of googling on this brought up surprisingly few good explanations as to exactly why it was going on. There are, of course, a lot of voodoo, crap explanations all over discussion boards, but little in the way of concrete evidence. Said voodoo even mentioned in this blog post as deadends describing the exact same thing I was experiencing. It wasn’t too long before I stumbled across Debian Bug report #301702 with a host of duplicates all describing the same problem but with little solidfying something such as a repro case, so I set out to do just that - repro the problem. Here’s how I did it.

First of all, cron quickly points to the following logrotate configuration (in package apache2.2-common or some variant) as a good starting point for investigation:
/var/log/apache2/*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f /var/run/apache2.pid ]; then
/etc/init.d/apache2 restart &gt; /dev/null
fi
endscript
}
But what on earth is going on such that Apache bails on a restart? In all of the other development and test environments we use, including production, this was only happening in a single one. The answer turns out to be that the problem is load related, or at least that’s how I was able to repro the problem. Slow apache children in general, however, are the cause of the problem.

I could not get this to repro simply by running the logrotate cron tasks as they would run normally (except for the –force, of course):
$ sudo /usr/sbin/logrotate --verbose --force /etc/logrotate.conf
or
$ sudo /usr/sbin/logrotate --verbose --force /etc/logrotate.d/apache2
In the above cases, apache would restart with no problems. So I introduced some load onto the machine. I did this basically by running updatedb (because this is also cron’d to run at the same time, in my case) as well as forking off a couple of while(1){}’s to use a bit of cpu.

With the introduction of load, I could repro the failure of apache to start during the logrotate, but I still didn’t understand what was going on with the log files. I had tunnel vision on the “Unable to open logs” red herring from the original cron complaint. Turns out the real problem is just a slow child and arguably poor behavior of the apache2 init.d scripts as well as the etch logrotate.d/apache2 configuration posted above.

In the repro scenario, I noticed apache complaining about slow children just before SIGKILLing them:
[Wed Jan 02 21:19:55 2008] [notice] Apache/2.2.3 configured -- resuming normal operations
[Wed Jan 02 21:20:15 2008] [warn] child process 18762 still did not exit, sending a SIGTERM
[Wed Jan 02 21:20:19 2008] [warn] child process 18766 still did not exit, sending a SIGTERM
... snip ...
[Wed Jan 02 21:20:21 2008] [error] child process 18765 still did not exit, sending a SIGKILL
[Wed Jan 02 21:20:22 2008] [error] could not make child process 18765 exit, attempting to continue anyway
[Wed Jan 02 21:20:22 2008] [notice] caught SIGTERM, shutting down

The introduction of load was causing children to be delayed in exiting. The /etc/init.d/apache2 scripts in etch attempt to kill the main apache2 process and once it is ascertained that it can no longer receive a signal (via kill -0, such as if the process is already gone), the init scripts continue on their way and attempt to start apache back up. At this point, they assume that apache and all of its children are gone, which is not necessarily the case. This is where the real problem lies. Long story short, those children which haven’t exited quite yet are still holding onto their sockets. As you can see in a trivial example where you try to run apache when it’s already running:
rwoodrum@slard:~$ sudo apache2
(98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs

There’s that pesky “Unable to open logs” message that was throwing my feeble brain off course.

The solution? A couple of things…

  • In Debian lenny/sid+, the logrotate.d/apache2 package config no longer calls /etc/init.d/apache2 restart and instead uses the graceful restart by way of /etc/init.d/apache2 reload. Apache is able to function much more cleanly in this fashion by holding onto its connections, not bailing on errors, etc.
  • In Debian lenny/sid+ the /etc/init.d/apache2 init scripts have been modified (hacked?) to add a long sleep time between the stop and start. Certainly not foolproof, but apache2 really should be able to exit in 10 whole seconds.

So now I have satisifed my who, what, when, where, and why.

Interview with Trilby Lundberg: ignorant, arrogant bitch

As one may surmise from the title of the post, I’m not a fan. Who is Trilby Lundberg? She publishes the Lundberg Survey, “a national survey of gas prices quoted regularly by major news organizations[.]” CNN interviewed Trilby (with an apparently Trilobite sized brain) in their “Fueling America” segment quite some time ago, and I saved it for the creation of this blog to write about. If you’d like to be infuriated, read CNN’s interview with Trilby Lundberg.

Some of the more prized gems of the interview?

I don’t accept [global warming] as established fact, nor do I accept that it would be caused by petroleum consumption, nor do I accept that the human species should not affect its environment. So even if it were someday to be shown to have some small effect on the environment, I see no crime. In fact, taking into account the many, many millions of people around the world that envy our way of life, it would seem more humanitarian to wish them the kind of plentiful petroleum products and vehicles … that we enjoy … to lift themselves out of [a] backward, poor way of life.

And certainly don’t forget:

I’m hoping that consumers will see through the rhetoric about consuming less, demanding less, as faulty. It is not a given that consuming less will be good for our economy or for our personal freedom. It is not even established for our environment that we [should] deprive ourselves of gasoline for our personal mobility as well our commerce. And to suppose that it is good to do that, and pretend that we have consensus and put our heads together to deprive ourselves of this great product that makes the country go around, commercially and individually, I think is flawed. I’m hoping consumers and voters will see through that and be able to ignore some of the most extreme suggestions.

Are you kidding me? Trilby, you poor soul, the planet we live on and the precious market you speak of are not one in the same. In the above comments the two have essentially been equated. (I’m betting Trilby is a hardcore Republican.)

Trilby does not accept as fact “that the human species should not affect its environment.” But wait, only in a pejorative fashion? Some pretty simple reasoning would seem to indicate that if we can impact it negatively, we should also impact it positively.

I also like the part about “consuming less” not necessarily being “good” for our “personal freedom.” It’s this kind of fearmongering that the Republicans have been championing since the unfortunate events of September 11, 2001. Is it just me or is Trilby basically sanctifying our dependence on foreign oil?

And isn’t it grand that Trilby’s solution to helping all those poor people in developing countries out of their “backward, poor way of life” is to use more oil. A veritable fountain of wisdom.

I’m sorry, but it’s this kind of ignorance and arrogance that have left us with a deforested and polluted planet. The solution is not to burn more petroleum. The solution is to rethink energy. I was fortunate enough to have CNN post my submission to a question about fuel consumption. As I said there and as I say again:

The problem with oil companies is just that — for too long they have thought of themselves as oil companies. They are not oil companies. They are energy companies. It’s time to move on.