Yes, I’m still alive

Hey all, sorry for the lack of posts. I hadn’t intended to go two months without posting. I suppose this makes it more of a rocky start than a bumpy one.

I thought I’d give a status update on where I am with things:

An update to Picture Switcher is about 88% done.
An update to Web Control is about 60% done.
An update to Desktop Log is about 40% done.
An update to Pref Setter is about 10% done.

As you may guess from the above, I’m spending most of my time with Picture Switcher. It’s keeping me rather busy, which has contributed to the lack of posts (though I’ll be honest and say not totally).

If it helps any, my friends are all wondering where I’ve disappeared to as well. ;)

Comments Fixed

Oops, my comment system somehow was a bit broken. It’s fixed now.

Sorry ’bout that, everyone.

Of Kqueues and Max Open Files

Kqueues are a great thing. They let you keep an eye on files, sockets — anything that has a file descriptor. It takes very little code, and you don’t have to worry about manually polling or the like. Awesome stuff.

There is one thing you do have to worry about, though. Your app’s number of open files. I ran into this problem in a spectacularly bad way. When I first released my application Picture Switcher (it’s a status menu item, which you can use to switch your desktop picture(s)), I had it going through the file system and checking the modification dates of the folders containing the pictures Picture Switcher knows about — when the user clicked on the status menu. This was a Really Bad Way to do things — it made the menu take a while to actually show (and if the user had a lot of folders, it could show the SPOD). So, when I was working on version 1.1 I decided to use kqueues to watch the folders instead of going through them when the user opened the menu. It was great, I didn’t need to worry about manually looking through all the folders anymore.

I get that all fixed up, and release 1.1 into the wild. And the crash reports started pouring in. I was genuinely puzzled by the reports — it had Picture Switcher crashing while trying to load a system framework. After looking at the crash logs, I thought maybe I had some sort of incompatibility with some haxies. The users who were sending me the crash reports all had quite a few of them installed. So, I began installing haxies on my system. One after another after another. And I still couldn’t reproduce the crash. I finally decided to make up a version that logged quite a bit of debugging info to a file on the Desktop, and sent that to a few of the people who were trying to help me figure it all out. This was after a few weeks had gone by — yes, I probably should have tried that earlier. I began to see what was going on, though I still didn’t know why. I saw that it was failing to open some of the file descriptors for the files, and then crashing a bit later when it tried to load a system framework. Finally, I figured it out by meticulously reproducing a user’s folder hierarchy on my system — I was hitting the max number of file descriptors I could have open and Picture Switcher was crashing after that because it couldn’t load anymore. I came to find out that system frameworks and plugins and such count as open files, and then on top of that I was keeping a file descriptor open on the user’s folders.

It didn’t take me long to search and find out how to raise the number of file descriptors I could have open. A simple call to setrlimit() would up my max number from the default 256 to the system defined max number of file descriptors an app can have open (by default, it’s at 10240 right now). And so finally, I implemented that and released version 1.1.1 (almost a month after the previous release). My users were happy, I was happy, and I learned it was a Really Good Idea to do a bit more testing before releasing, especially on something variable like this.

Do any of y’all see the flaw there? If not, don’t feel bad — I didn’t either until just recently. Or rather had it come back at me.

A few weeks ago, a user emailed me to tell me that Picture Switcher was crashing on launch. I didn’t get a crash report with it, but immediately I thought “Hmm, this seems like the max open files crashing bug.” So I emailed him back, asking if he was using the current version and asking for the crash report. He emailed me back, letting me know that he was indeed using the latest version, and also attached the crash log. Looking at it, it seemed at first that it was a different bug — the code was crashing in a method where I get the display names. So I whipped up a test version to make sure I didn’t have a bug there, and asked him to run it and send me the console log. He did, and lo and behold, there in the console output, were a few messages letting me know that it was indeed the max open files bug. I had added a log message that output an error letting me know it couldn’t open any more file descriptors when I was debugging this and had forgotten to take it out — a good thing in this case.

My immediate response to that was “Ack, it’s the specter of the max open files raising its ugly head at me again!” At first, I thought that maybe the setrlimit() call was failing on his machine for some reason, and sent him another test version that logged it. But no, it was working fine. This, of course, made me realize that I hadn’t totally thought things through the last time. 10240 file descriptors seems like a lot, but obviously it is a limit and I should have thought about that during the last release. All I can say to that is mea culpa — I had worked a month trying to figure it out and just wanted to get the fix out there. I’m sure some of y’all have been in that situation. But still: Bad programmer, bad!

I sat there for a while thinking through this. I realized this was a really bad thing from the get go. While I could raise the limit of open files, I really hadn’t thought what that might do to the system. The system has a max number of open files for the entire system — set at 12288. That’s only 2,048 files more than what I set. So, actually thinking that through, I realized that if someone had enough folders that Picture Switcher is watching, I could potentially greatly limit the rest of the system’s resources here. Doing things like that is something akin to using a sledge hammer to put a tack in the wall — way overkill, and potentially damaging. Not a nice thing to do. Bad programmer, again.

This time, I set down some ground rules for myself to follow. 1) Don’t raise my number of open files to the max, 2) figure out some way to know when I’m about to hit that number, and 3) bring back manual polling if I do go over that number. Just a bit better this time.

After doing a lot of searching, I found out it’s not trivial to know the number of open files you have. I then decided to download the source to lsof from the Darwin source code to see how it’s done. I have to admit, it was a bit beyond me. So I needed to figure something else out. Finally, I came up with a solution. It’s not 100% foolproof, but I did do a bit of coding to fix things up if it doesn’t work. Here’s my solution.

I raise the max file descriptors to the max per process divided by 10, plus 300. That puts me at 1340, by default. The added 300 is a bit of padding so that I have extra descriptors around when I need to open things, and for the system frameworks and plugins that are loaded. In my header file, I added an instance variable for maxFileDescriptor, then used this code in the init method of the class:

struct rlimit myLimit; int mib[2], maxOpenPerProc = 0; size_t inputSize = 0; mib[0] = CTL_KERN; mib[1] = KERN_MAXFILESPERPROC; inputSize = sizeof(maxOpenPerProc); sysctl(mib, 2, &maxOpenPerProc, &inputSize, NULL, 0); myLimit.rlim_cur = myLimit.rlim_max = (maxOpenPerProc / 10) + 300; setrlimit( RLIMIT_NOFILE, &myLimit ); maxFileDescriptor = getdtablesize(); maxFileDescriptor -= 300; if ( maxFileDescriptor < 0 )     maxFileDescriptor = 0;

In the method I call to add items to the kqueue, I test to see if maxFileDescriptor is a valid file descriptor. This is a sort of kludge, but it does work. It’s not 100% foolproof, as I mentioned before, because there’s a possibility that there were more than 300 system frameworks and plugins that got loaded, which don’t get a file descriptor and hence this test would report a false negative. This is a slight possibility, but I’m not taking chances this time.

if ( isatty( maxFileDescriptor) || errno == ENOTTY ) {     /* Code to add it to a manual poll method */ } else {     /* Add it to the kqueue */ }

What isatty() does is check to see if maxFileDescriptor is a TTY. This isn’t ever going to be the case with Picture Switcher, but isatty() will set the global errno to ENOTTY if the file descriptor is valid but not a TTY (otherwise, it sets errno to EBADF).

If the check reports a negative, but I can’t open the file descriptor, I do a bit of cleaning — I add it to the manual poll method, then I remove a few of the items from the kqueue and add those to the manual poll method as well to free up a few file descriptors.

So for any of y’all out there using kqueues, do keep an eye on your open files count (you can do this with lsof in Terminal, or with Activity Monitor). If you’re even coming close to the max open files you have set (256 by default, if you haven’t changed it), you may well want to do something similar in your code.

Welcome to Ramblings of a Mad Poet

I’ve been thinking on and off over the last few… years, really… of starting up a blog. It’s a great way to keep in touch with my users, and gives me a voice in the developer world — something I have been wanting to do. So why has it taken me so long?

Well, the short and long of it is is that I’m a quiet person. I’m not much given to talking or writing something without it being a reply to someone else talking or writing to me. I enjoy reading blogs, but the thought of writing my own just rubbed me the wrong way. It’s somewhat contrary to my nature. There’s also the issue where I really don’t do a lot of exciting things, and I’ve often thought that any blog that I wrote would be a great way for an insomniac to have a way to go to sleep. :smiley2_tb:

So what changed my mind?

Y’all did. You, my users. Not from anyone emailing me to say “You should have a blog” or “Why don’t you write about things?” but from emails asking about if I’m still working on such and such an app, or when might I have an update out, or even emails just saying “Thanks for this great app.” I tend to go anywhere from 6 to 8 months between releases on Pref Setter, and longer with some of my other software. I don’t really put up anything about my development cycle while I’m at it. Mainly because I haven’t had a place to do so. Also, as I stated above, I’d like to have a voice out there with the other developers. Not because I want to be known or any such thing; I’d just like to contribute. Every voice added makes the choir sound that much sweeter.

And so here I am, starting out my very own blog. I hope y’all will stick around for it — it may be a bit of a bumpy start while I get used to writing and finding my style. I’m not promising to have something posted every day, or even every week, but I’ll do my best to get over my quiet nature and post something at least somewhat often. My plans are to mostly post about things that come up while I’m coding this or that, but I may occasionally post something off that topic. I’m not really planning on doing that often — this is tied to my software site, after all.

So, I thought I’d speak a bit about the blog itself, this being my first post and all. Why’d I name it Ramblings of a Mad Poet? Truth to tell, I tend to to ramble a bit when I write. Most likely because I don’t do a whole lot of it, and also because I’m breaking through my quiet nature and that sort of unblocks a torrent of words. So what’s with the ‘Mad Poet’ part? Honestly, I’ve always just liked that phrase. A mad poet. It evokes a picture of some guy hunched over a table full of paper scraps, writing feverishly and unable to stop. Something like I do when I actually sit down to write. But, also, I actually am a poet. There’s an old (now defunct, hasn’t-been-updated-since-October-2000) site where I used to put up some of my poetry. If you do a search for “Darkshadow,” you’ll turn it up if you’re really that interested. I stopped posting there after Yahoo took over Geocities. Things changed, and also I was flagging a bit on the enthusiasm side on posting my poetry. I still write some here and there, but I don’t post it anymore. And, I actually do consider code as a sort of poetry. That’s nothing new there — lots of developers have stated that. But hey, real poet here, and I concur. Code is poetry. :happy_tb:

I hope y’all enjoy the design of my blog. I know it jars somewhat with my main site design, but I wanted something a bit more personable for my blog. The main design didn’t really lend itself to that, so I decided on this design. Interestingly enough, I actually originally came up with this design two years ago. I had a (brief) interest in starting back up my poetry site. This isn’t exactly the same design, but it’s very close to what I did for that. The design really fit perfectly into what I wanted for my blog. So a loss for the poetry site (if I ever decide to bring it back), but a win for the blog.

A final question for y’all reading this — what sort of things would you like to see me post? This early in things, I’m wide open to suggestions.