Tripoli: Differential code coverage tool

Posted in Eclipse, Hacking, programmer productivity at 6:41 pm by ducky

In the observations of professional programmers that I did for my thesis, I frequently saw them get sidetracked because they didn’t have good information about whether the code they were looking at was executed as part of the feature they were exploring. I theorized that being able to look at the differences in code coverage between two runs — one where they exercised the feature and one where they didn’t — would be useful. Being able to see which methods executed in the first but not the second would hopefully help developers focus in quickly on the methods that were important to their task.

In my last month at UBC, I whipped up a prototype, an Eclipse plug-in called Tripoli, and ran a quickie user study. The differential code coverage information really does make a big difference. You can read a detailed account in UBC Technical Report TR-2008-14, but basically graduate students using Tripoli were frequently more successful than professional programmers who didn’t use Tripoli.

As an example, in the Size task, the subject needed to find where a pane was resized. That was hard. There is no menu item or key stroke invocation for developers to use a starting point. They also didn’t know what the pane was called: was it a canvas, a pane, a panel, a frame, a drawing, or a view? Between pilot subjects and actual subjects, at least eleven people tried and failed to find a method that was always called when a pane was resized and never called when it wasn’t. I even spent a few hours trying to figure out where the pane was resized and couldn’t.

As soon as I got Tripoli working, I tried the Size task, and in my first diff, I saw ten methods, including one named endResizingFrame(). And yes, it turned out to be the right spot. I realized that most of the methods came from not hovering over the corner of the frame such that the cursor changed into the grab handle, and reran.  This time I got exactly one method: endResizingFrame().  Wow. The graduate students in my second study were also all able to find endResizingFrame() by using Tripoli.

Tripoli does take some getting used to. Even as the author, it took me a while before I consistently remembered right away that I could use Tripoli on my problems. I’ve also clearly gotten better over time at figuring out what to do in my second run to ensure that the “diff” has very few methods in it.

If you want to try Tripoli out, I’ve posted it online. Just remember it’s a prototype.


geek cool alert: Triage

Posted in Hacking, programmer productivity, Technology trends, Uncategorized at 11:01 am by ducky

There’s a cool paper on a tool to do semi-automatic debugging: Triage: diagnosing production run failures at the user’s site. While Triage was designed to diagnose bugs at a customer site (where the software developers don’t have access to either the configuration or the data), I think a similar tool would be very valuable even for debugging in-house.

They use a number of different techniques to debug C++ code.

  • Checkpoint the code at a number of steps.
  • Attempt to reproduce the bug.  This tells whether it is deterministic or not.
  • Analyzes the memory by walking the heap and stack to find possible corruptions.
  • Roll back to previous checkpoints and rerun, looking for buffer overflows, dangling pointers, double frees, data races, semantic bugs, etc.
  • Fuzz the inputs: intentionally vary the inputs, thread scheduling, memory layouts, signal delivery, and even control flows and memory states to narrow the conditions that trigger the failure for easy reproduction
  • Compare the code paths from failing replays and non-failing replays to determine what code was involved in that failure.
  • Generate a report.  This gives information on the failure and a suggestion of which lines to look at to fix it.

They did a user study and found that programmers took 45% less time to debug when they used Triage than when they didn’t for “real” bugs, and 18% for “toy” bugs.  (“…although Triage still helped, the effect was not as large since the toy bugs are very simple and straightforward to diagnose even without Triage.”)

It looks like the subjects were given the Triage bug reports before they started work, so the time that it takes to run Triage wasn’t factored into the time it took.  The time it took Triage to run was significant (up to 64 min for one of the bugs), but presumably the Triage run would be done in background.  I could set up Triage to run while I went to lunch, for example.

This looks cool.


it's not just me!

Posted in Hacking, Random thoughts at 8:31 pm by ducky

It’s not just me! Someone else gets frustrated by Web sites that won’t allow dashes or spaces in credit card numbers!

It is just unfathomable to me why any Web site would not insert the ONE LINE OF CODE to handle spaces and dashes. I am glad to see that it’s not just me.


geek humour

Posted in Family, Hacking, Random thoughts at 9:38 am by ducky

My husband and I are geeks. This manifests itself in many ways. One way is that when we moved up to Canada from Palo Alto, we numbered all of the boxes and logged all of the contents of all of the boxes.

In anticipation of our move to a tiny tiny apartment in downtown Vancouver, I packed up a box of books and class notes to take down to our storage locker in the US. Jim said that he’d been assigning new boxes numbers in the 200 series — 200, 201, etc.

Me: “Jim, would you be a name service for box numbers?”

He pulled out his PDA and got ready.

Me: “Um, ‘Hello.'”

Jim said nothing but was suppressing a grin. He continued to say nothing.

Me: “Doh! Right! Carriage-return, carriage-return!”

Much laughter ensued. We are such geeks. 🙂

(When taking DIRECTLY to Web servers, i.e. not through a Web browser, you have to issue a command like “GET / HTTP/1.0” and follow it with *two* carriage returns. One won’t do, and it’s a really easy mistake to make.)

(PS, Yes, I know that HELOs (used in email protocols) don’t need two carriage returns.)

(PPS, Yes, I know that technically it’s CRLF, CRLF, not CR, CR.)


help? tab spam in different IDEs?

Posted in Eclipse, Hacking, Technology trends at 5:42 pm by ducky

There are a zillion graphical IDEs out there, and I really don’t want to download and try each one. I don’t even want to try twenty of them. So, dear readers, care to help me?

All the IDEs that I’ve seen have a main center panel with the source of the file you’re looking at. Above that, they seem to all have a row of tabs, one per file that you’ve opened. (Does anybody have anything different?)

Here is a link to a screenshot of Eclipse. (Sorry, it was too wide for my blog.) Eclipse puts a little “>>” to show that there are more files (as pointed to by the ugly black arrow), with the number of hidden tabs next to it (“4” in this case). Clicking on the “>>4” gives a drop-down menu (shown here in yellow).

What happens in other IDEs when you have more tabs than will fit across the horizontal width of the source file? How does your IDE handle it? Or does your IDE have a different tabbing model altogether, e.g. it opens one window per file?  I would greatly appreciate hearing from you.

You can either email me ducky at webfoot dot com or post in the comments; I’ll wait a few days and then summarize in another blog posting.  Remember to tell me the name of your IDE.


robobait: how to turn off microphone in Windows XP on ThinkPad

Posted in Hacking, robobait at 9:24 am by ducky

To turn off the sound on a ThinkPad running Windows XP:


Control Panel->

Adjust the system volume ->

In the Device volume area in the middle of the Volume tab, click on Advanced. I know, I know, it looks like you are adjusting speaker volume, but go ahead and do it.

If the microphone doesn’t show up as one of the devices, select the menu item Options->Properties and put a check in the Microphone box.

In the microphone sub-panel, put a check in the Mute box.

(I am soooo not a Windows person…)


open source funding models

Posted in Hacking, Random thoughts at 9:05 pm by ducky

Paul Ramsey posted recently that open source was funded by four basic types:

  • The altruist / tinkerer
  • The service provider
  • The systems integrator
  • The company man

He continued by saying, “Notably missing from this list is ‘the billionaire’ and ‘the venture capitalist’.”

I have to quibble a little bit.

Mitch Kapor personally financed (most of) the Open Source Applications Foundation, which pumped quite a bit of money into open source projects.

Obviously a lot of it went into the Chandler project, but some went into various framework projects in order to make them work well enough to use.  For example, OSAF supported someone full-time for several years to improve the Mac version of WxWidgets.

There is also a fair amount of work done by people who made a ton of money and are now tinkering.  I know personally a few people who made boodles on IPOs, retired, and now spend their time on open-source projects.  While you could say that these are in Paul’s “tinkerer” class, these people can invest a lot more energy than someone on nights and weekends is likely to.

I will acknowledge, however, that the fraction of open source work financed by rich folks is probably small.

However, I think Paul is missing some significant sources of open source financing.  Paul’s definition of “company man” talks of people who have a little bit of discretionary time at their work.  Maybe it’s different outside of Silicon Valley, but I don’t know many people who have much discretionary time at work.  On the other hand, there are a fair number of people whose work leads them to contribute to open source in the course of their work.

  • Sometimes they use an open-source project in their work, find that it has some bugs that they need to fix to make their project work, and contribute the fixes back.  This is self-interest: they would rather not port their fixes into each new rev of the open-source project.  My husband said that his former company contributed some fixes for Tk for this reason.
  • There are a few companies who use a technology so much that they feel it worthwhile to support work on that technology.  Google pays Guido von Rossum’s salary, for example.
  • I don’t know what category to put Mozilla into, but it makes money from partnerships (i.e. Google) to finance its development.
  • There is a non-zero amount of money that goes into open source — directly or indirectly — from governments and other non-profit funding agencies.  For example, the Common Solutions Group gave a USD$1.25M to OSAF; the Mellon Foundation gave USD$1.6M.  This made perfect sense: it is far cheaper for university consortia to give OSAF a few million dollars to develop an open-source calendar than it is to give Microsoft tens of millions every year for Exchange.
    • While I don’t have hard data, I bet that a fair amount of open-source work gets done at universities.  All of my CS grad student colleagues work with open source because it’s cheap and easy to publish with.  While they might not release entire projects, I bet I’m not the only one who has fed bug fixes back to the project.


seeking story source

Posted in Hacking at 12:29 pm by ducky

I heard a story once, but don’t have any idea where it came from. It’s too good a story to let die, so do any of you know where this story came from?

A computer science department had a programming competition of some sort. The winner would be the team that computed the result fastest. The contestants could enter as many times as they wanted without penalty.

One team entered a (slow) program that calculated the value, wrote it to a file, submitted it, and quit. They then entered a second program which read the value from that file and submitted it. They were way, way faster than any of the other teams.

The computer science department was split. About a third didn’t care. A third thought that the team were cheaters and should be completely disqualified and scorn heaped upon them. A third thought that the team was brilliant and should get the top prize and high praises.

The way I remember the story, the department was ripped apart by this issue. It laid bare some fundamental differences in value priorities such that the faculty was unable to work together.

Is this apocryphal? Urban legend? True? Help me out here….


Seeking professional Java programmers!

Posted in Hacking, University life at 10:54 am by ducky

Help improve software tools by participating in a user study!

I am looking for Java/Eclipse programmers to participate in a study on how developers navigate through code. If you are interested in participating, or know someone who might be, please contact me at duckys at cs.ubc.ca . (Do not use my ducky at webfoot address.)

I have assembled some tools that I think can make navigating through code easier, and want to see how actual developers navigate with and without those tools. I need pairs of professional developers who who have programed regularly in the Java using an Eclipse-based integrated development environment (like Rational Application Developer for WebSphere or JBuilder) for at least the past six months. You need to be over 19 and proficient in English.

If you have a buddy that you could team with, fantastic! If you are solo, not to worry – I can find someone for you to pair with. People in the Vancouver, BC area are easiest for me to work with, but I can travel to Vancouver Island or Washington State if that’s what it takes.

Participating in this study will take approximately 3 hours of your time. It involves meeting me and one other participant at a mutually-agreeable location to perform four coding tasks on a hw/sw system that I will provide.

You and the other participant will pair-program for four coding tasks: two with a pretty “normal” Eclipse and one with enhanced navigation tools. Both versions of Eclipse will log your interactions; I’ll also do screen capture, and record the conversations you have with your partner. Afterwards, I’ll ask you to fill out a brief questionnaire, and briefly interview you about your experience.

We might also offer you the option of continuing to use the tool in your normal day-today work. If you choose to do so, we’d appreciate getting feedback from you afterwards.

In exchange, I can offer a $20 Amazon.ca gift certificate (w00t!). If you’re interested, I’d also be delighted to tell you what I’ve learned about programmer productivity.

I am conducting this study as a part of my MS Thesis, supervised by Dr. Gail Murphy. This study is being funded by the National Sciences and Engineering Research Council of Canada.
I hope to hear from you!

Kaitlin Duck Sherwood


difficult choices

Posted in Hacking at 5:03 pm by ducky

I’m working on making a version of Eclipse that has different tabbing behaviour. If someone does the following in the PackageExplorer, I want the following to happen:

  • double click on file – open file in the same tab
  • single click on file – deselect (dehighlight) previously selected files, select (highlight) that file
  • shift-control-click – deselect previously selected files, select the file, and open it in a new tab
  • control-click – add file to selected files

Right now, I can’t get both control-click and shift-control click working this way. Either I don’t get control-click at all, or shift-control-click adds the clicked-upon file to the selection list. The latter is not only visually wrong, but causes problems — when I try to open the selection, it opens ALL the selected (highlighted) files.

I can instead tell Eclipse that the PackageExplorer doesn’t support multi-select, in which case control-click will act just like a single click.

I went looking for where the control+click is examined, to put in a line that says “if shift is down, do a single selection and not a multi-selection”. Unfortunately, I had to go all the way down into the bowels of GTK. It looks like the place I want is in gtktreeview.c, in gtk_tree_view_button_press(), in the vicinity of line 2573 in GTK+-2.10 (which is probably not the exact same version as is in the Eclipse I’m running).

Now I have a dilemma. Do I go through the pain and agony of building my own GTK, then building my own Eclipse, in order to get the “right” behaviour? Or do I bail and give up the control-click multiple-selection functionality?

Neither. I go home and think about it.

« Previous Page« Previous entries « Previous Page · Next Page » Next entries »Next Page »