Archive for October, 2008

Notes on “Augmenting the Social Space of an Academic Conference”

October 27, 2008

Augmenting the Social Space of an Academic Conference: Joseph F McCarthy, David W McDonald, Suzanne Soroczak, David H Nguyen, Al M Rashid

The authors built and systematically evaluated two Ubicomp applications. The applications were designed to augment an academic conference. Both relied on RFID tags in delegate conference passes.

The first system, AutoSpeakerID, projects the delegate’s profile on large screen when the delegate asks a question of a conference speaker.

The second system, Ticket2Talk, displays delegate profiles on a large screen as they interact in a coffee area.

The authors evaluated both systems by observing their use and surveying conference delegates for their thoughts.

Both systems are interesting additions to a conference. They are both simple to build. The evaluations suggested there is some value in them, even if it is only novelty value.

In general, I’m skeptical of any technology system designed to cause people to interact where they otherwise wouldn’t. In my opinion, the Ticket2Talk system fits in this category. I don’t understand why so many researchers keep trying to build systems to get people to talk to strangers more.

Advertisements

Notes on “The Familiar Stranger”

October 26, 2008

The Familiar Stranger: Anxiety, Comfort and Play in Public Places: Eric Paulos, Elizabeth Goodman

A familiar stranger is someone we recognize but we don’t speak with. The paper explores how emergent wireless technologies might affect our lives and interactions with these familiar strangers. The authors conducted two experiments and presented one ubiquitous computing device concept.

In the first experiment, they photographed everyone present at a train station at a certain time. A week later, they returned and asked everyone at the train station to identify which photographs they recognized. They also distributed a survey about public relationships.

In the second experiment, they walked around Berkeley with a test subject and interviewed him or her about significant places, comfort levels and relationships.

The proposed device transmits an identifier and listens for others of its kind. In this way it leaves a “digital scent” and smells for other digital scents left by others. Repeated detection of the same digital scent implies the existence of a familiar stranger.

The paper’s conclusion makes the following claims: 

The very essence of place and community are being redefined by personal wireless digital tools that transcend traditional physical constrains [sic] of time and space. New metaphors for visualizing, interacting, and interpreting the real-time ebb and flow of urban spaces will emerge. Crucial to this discussion will be the often ignored yet vital role of our Familiar Strangers. Without a concerted effort to develop new knowledge and tools for understanding the implications of these new technologies, computer and social scientists, city planners, and others run the risk of losing touch with the reality of our urban streets and their inhabitants.

If the authors are right about the assumptions made in these statements, it is conceivable that informal foundational work will be useful. In the end, however, this is a paper grounded in art more than in science. Books and art pieces dominate the citations list. The experiments consist of surveys unscientifically exploring the obvious. There’s not much here.

I don’t get REST

October 25, 2008

I thought I understood REST to mean “Name your nouns with unique URLs and use HTTP verbs to modify and access them.” I learned this through Ruby on Rails blogs and books when REST was the fad du jour. I remember thinking that all the explanations I could find of REST were too complex and that it was a simple idea at its core. I didn’t understand why the complexity was there but I didn’t think too much about it.

Along came an article from REST’s inventor expressing frustration at things being called REST when they don’t match his definition.

It appears that “REST” is a word with two distinct meanings. One meaning is as described above. I’m not alone, as others use the same interpretation. Roy T. Fielding holds the other meaning. It’s clear that there are deep ideas and subtleties in what Roy originally meant by REST. It’s also clear that Roy has difficulty communicating these ideas. Some evidence: The comments on the blog post, a link on Hacker News, Sam Ruby’s take. I certainly don’t understand Roy’s comments and I can’t even say whether or not Roy’s REST is actually coherent.

Midterm TA Evaluation Results

October 23, 2008

I TA CSC 290. Today I asked the students to evaluate me using a form I created.

I asked seven questions with Likert scales and also solicited free-form comments.

CSC 290 Midterm TA Evaluation Form

CSC 290 Midterm TA Evaluation Form

I received 21 responses. The class has 24 students.

Full results below.

 

TA Evaluation Results Chart & Graph

TA Evaluation Results Table and Graphs

October 21 Code Reading Meeting

October 22, 2008

We met in the DGP lounge (Projector that speaks DVI! Yes!) for an hour an went over the Django URL dispatch code. We talked about how Python is a good language for this group’s purposes.

We also debated the ideal format for a code reading group—we’ll get a good system worked out. The tradeoff is that it’s better for us all if we each go over some assigned code individually, but not everyone has the time to do it.

For next week (Tuesday, October 28, 5pm, BA 5181), we’re going to look at BitTorrent. We’ll collaboratively explore it at the meeting but those of us reading ahead should think about:

  • What optimizations could be put in for the one-server one-client case?

Before the meeting, everyone will send out their preferred tools, tricks and techniques for exploring an unfamiliar piece of Python code.

For my part, I like to browse around using a graphical debugger and an architecture diagram where possible but for now I’m stuck with TextMate and PDB.

Scopes and the Properties Pattern

October 20, 2008

Background

Read this first. It’ll take about half an hour, but it’s worth it. Note that it in turn recommends you read Godel, Escher, Bach: An Eternal Golden Braid, which is a six-month project, but worth it too. Come back when you’re ready.

Now, I’m very lucky to have help understanding the article. I learned a little bit of JavaScript back in the day, so I already knew about the Prototype Pattern. I even knew about the “Array-Map surjection” even though I didn’t know that’s what it was called.

Wait, you don’t know about the Array-Map surjection? You didn’t read the article, did you? Go back and read it!

Idea Seed

One day I was idly dreaming about the Properties Pattern and it occurred to me that scopes and symbol tables use the pattern in most programming languages. A variable declaration is a “put.” A defined? (in languages that support it) is a “has.” A variable access is a “get.” Function calls introduce a new Property List and set the parent to the current scope. Undef (in languages that support it) is a “remove.”

I thought about whether it would be useful to make this explicit. It occurred to me that “big differences” between programming language constructs, like between dynamic and lexical scoping and between functions and macros, are really small differences in how the property lists are being modified. Maybe we could make a programming language that gave us power over these things.

Pre-emptive Defensive Digression

Stop right there. You’re thinking about all the things that are wrong with the idea. You’re thinking about all the similar, failed projects with similar concepts. Work with me here. Let’s get into Yes, Yes mode. Think about “How can we make this meaningful and good?” 

I’ll admit I don’t know anything about programming languages. Maybe this is basic stuff that I’m stupidly rediscovering. Maybe this is a dangerously fuzzy idea that can never be made concrete. I’m having fun thinking about it. Go read someone smarter’s blog.

Examples

Let’s look at scopes as instances of the properties pattern. When you define a new function, you create a new empty property list (“symbol table”) with its parent set to the current property list. 

Here’s the difference between lexical and dynamic scoping. In a lexically-scoped language, a function call takes the function and leaves its parent alone. In a dynamic language, a function call sets the function’s parent to be the caller’s scope. That’s it.

What if you could do f.scope.parent := currentScope()? What if you could choose whether to call functions lexically or dynamically depending on the problem at hand? What if you could do f.scope.parent := thatScopeOverThere()?

I don’t know either.

What’s a function call, really, if you think of it as operations on variable names? Don’t think about mathematical formalisms for programming languages. Forget higher-level abstractions of machine instructions for a moment. A program is a bunch of operations on variable names and you make programming scale by tucking away those variable names into scopes. You control the scopes with “macros” (hygienic and non-hygienic and preprocessor) and “functions” and “objects” and “closures” and “threads” and “processes” and “continuations”. The point is, at some level of abstraction, we’re just playing with scopes and names. Object-oriented programming especially: What are “inheritance hierarchies”, “polymorphism” and “classes” and so on but rules and systems for playing with scopes?

What’s the difference between a macro call and a function call? Between a hygienic macro and a non-hygienic macro? They’re all just different ways of tweaking the scope/property lists. What if we made these tweaks user-accessible and explicit?

Help wanted.

Part Two

Or, While I’m on the Topic of Crazy Programming Language Ideas.

Look at those operations that form the crux of the Properties Pattern: get, put, delete and has.

They look a lot like those HTTP verbs we keep hearing about in the context of RESTful resources.

Hm. 

Let’s get crazy.

What if we make a “programming environment” where the “symbol table” is the Web Itself? Variables are HTTP URLs? Types are optional, and they’re MIME types? First-order functions are web services?


op +;        # http://ops.gnu.org/+
op /;         # http://ops.gnu.org/div
op reduce;  #http://ops.gnu.org/reduce
add_list = part_func_app(reduce(+,int(0)))

def_service len(*lst): 
#http://bigexpensivecompany.bad/len
    reduce(fn(accum new) (+ accum 1),0, lst)

def_service avg(*lst): 
#http://bigexpensivecompany.bad/avg
    /(add_list(lst), len)

def_service SQL(query,*db_connection): 
#http://bigexpensivecompany.bad/SQL
    /* elided */

average-manager-salary = 
  avg(SQL("SELECT salary FROM emp_salary", 
http://intranet.bec.bad/db/hr/managers.conn))
# http://bec.bad/careers/resources/average-manager-salary


GET /index.html HTTP/1.1
 Host: http://ops.gnu.org/avg?
    p1=
      http://bec.bad/SQL?
        p1=SELECT+salary+FROM+emp_salary&
        p2=
          http://intranet.bec.bad/db/hr/managers.conn

.
.
.

 HTTP/1.1 200 OK
 Date: Mon, 23 May 2005 22:38:34 GMT
 Server: Apache/1.3.3.7 (Unix)  (Red-Hat/Linux)
 Last-Modified: Mon, 23 May 2005 22:38:34 GMT
 Etag: "3f80f-1b6-3e1cb03b"
 Accept-Ranges: bytes
 Connection: close
 Content-Type: x-s-web/x-currency-xml; charset=UTF-8

<currency type="USD" 
          timestamp="Mon, 23 May 2005 22:38:34 GMT">
136400
</currency>

Causal vs. Effectual Reasoning

October 20, 2008

Cathartic Ideas

Every once in a while a piece of writing comes along that sums up a bunch of half-baked ideas and makes them clear. You read it and you think “That’s what I meant all along!” and “Yes! yes!” and “That’s why this and this and this!” and yet you never put it as elegantly or explicitly as the author. Call it a cathartic idea.

Saras D. Sarasvathy’s “What Makes Entrepreneurs Entrepreneurial” presents a cathartic idea. (Warning: PDF)

From the title, I expected a controlled study of attributes of entrepreneurs vs. non-entrepreneurs. It’s not. It’s about thinking styles, and how veteran entrepreneurs have a different thinking style from other businesspeople. 

Types of Problem Solving

Most of us use causal reasoning. We look at a problem and look at our resources and figure out how to use our resources to solve the problem. Logical, right?

Some people are smarter. They use creative causal reasoning. They look at a problem and look at their resources and figure out what new resources they can go get to solve the problem. Even better. Roger Martin hints at this sort of thinking in his book The Opposable Mind, thought the focus is on integrative thinking which is another matter entirely.

Sarasvathy claims that good entrepreneurs use effectual reasoning, especially when they start something new. Here, they look at their resources and the resources they can get and try to figure out what valuable problems they can solve. 

Effectual Reasoning in Practice

A bad budgeter thinks about all the things he or she wants and then tries to figure out how to get them. A good budgeter thinks about what he or she has got and tries to figure out what he or she can get with it.

A bad government looks at all of society’s problems and starts to look for solutions. A good government looks at the money and resources available and looks for the most important problems that can be solved.

A good artist is always out of the box and turns ordinary materials into powerful emotional artifacts.

Good researchers solve problems and answer questions. The best researchers invent new problems to solve and find new questions. Think of Claude Shannon inventing information theory or Mark Weiser’s vision for ubiquitous computing.

Thinking Styles Matter

For a long time I have been convinced that we need to pay more attention to problem solving and thinking style as skills to be improved. Everyone talks about university being about “learning how to think” but how do we quantify or validate that university education makes better thinkers? Which is better, a philosophy degree or a computer science degree? Math? History?

There are results in Cognitive Science that elaborate on how people solve problems, but only cognitive scientists have seen them. We can get better at solving problems in general but nobody teaches these skills. Some good books exist like Roger Allen’s Winnie the Pooh on Problem Solving but few have read them. How many people have had an evidence-based class on brainstorming and creativity techniques that talked about something better than mind-mapping? (I haven’t, but I’d like one.)

There’s a problem to be solved here.

Notes on “A Taxonomy of Ambient Information Systems: Four Patterns of Design”

October 19, 2008

A Taxonomy of Ambient Information Systems: Four Patterns of Design: Zachary Pousman, John Stasko.

It is useful to discover meaningful dimensions on which ambient information systems vary. With multiple dimensions identified, a space is created which reveals clusters of similar applications and holes. The clusters can be named to create a common vocabulary for discussion. The holes focus our attention on opportunities for innovation.

In this paper, the authors describe a four-dimensional classification system for ambient information systems. They locate 19 existing applications within this space and identify four clusters which they call “patterns.”

The four dimensions:

  1. Information Capacity
  2. Notification Level
  3. Representational Fidelity
  4. Aesthetic Emphasis
The authors don’t make any sort of comparison between their taxonomy and previous taxonomies. This might be considered a fault in the paper but I think the intuitive appeal of the given taxonomy is sufficient argument for its use. The classification system seems to match the important features of ambient information displays. I’m not too worried about the construct validity or face validity of the system.
That said, the authors ought to present rubrics and show some evidence that novice users of these rubrics classify applications in the same way. I would hope for high inter-rater reliability but I think they would need to construct their rubric carefully to achieve it.

Notes on “Heuristic Evaluation of Ambient Displays”

October 19, 2008

Heuristic Evaluation of Ambient Displays: Jennifer Mankoff, Anind K. Dey, Gary Hsieh, Julie Kientz, Scott Lederer, Morgan Ames.

Jakob Nielsen’s Ten Usability Heuristics are the gold standard of heuristic evaluation for computer interfaces. He has established their validity and utility for the improvement of software. In fact, he established heuristic evaluation itself as a legitimate technique.

His heuristics don’t intuitively apply to Ambient Displays, however, where the goals are different from traditional user interfaces. In a traditional user interface, the user’s full attention is focused on a task. An ambient display is meant to capture the user’s attention only when necessary and even then only for a brief moment.

The authors of this paper brainstormed new heuristics for the evaluation of Ambient Displays. They compared the new heuristics with Nielsen’s heuristics and concluded that the new heuristics were useful and superior to Nielsen’s.

The paper provided insight into how they went about developing their new heuristics. They also made a significant effort to attach a quantitative and empirical foundation for the new heuristics. This seems like a difficult obstacle to overcome given the fluid and subjective nature of heuristic evaluation in general.

I’m suspicious of their results. First, they failed to find an effect for the hypothesis that “the number of issues found in the ambient condition will be greater and the issues will be more severe than those found in the Nielsen condition.” Second, they eliminated an outlier to find an effect for the hypothesis “The ambient heuristics will be more useful to evaluators than Nielsen’s heuristics. A heuristic that finds many, severe problems is more useful than a heuristic that finds fewer problems with lower severity.” I don’t agree that the elimination of the “outlier” was appropriate or even that the data point they eliminated was an “outlier.”

The problem was that they weren’t making a valid comparison. Nielsen’s heuristics are inferior to any set of heuristics designed for ambient displays, because Nielsen’s heuristics aren’t about ambient displays. It’s like applying an evaluation chart for a beauty pageant to a dog show. I think a more appropriate experiment would be to instruct evaluators to find all the issues with an Ambient Display. In one condition, give them the heuristic list and in the control condition give them no list.

Notes on “Semantic Hashing”

October 13, 2008

Semantic Hashing: Ruslan Salakhutdinov, Geoffrey Hinton

I read this paper at George’s suggestion.

The gist: The authors present a way to create semantic hash codes for documents. The system outputs binary strings for documents such that similar documents result in strings with a low Hamming distance. They accomplish this by using a two step procedure for training a stack of Restricted Boltzmann Machines.

I like the promise of this paper, though I’m not convinced I could reproduce its results given only the information in the paper. Perhaps this is deliberate on the part of the authors, or perhaps it’s because I’m not a machine learning expert. I’m certainly tempted to try to build such a system and sell it to corporate intranet search companies.

I love that they trounce LSA. Once I wrote a rather long paper about LSA. I was rather taken with it at the time.

I am beginning to believe that dropping Hinton’s course was a bad judgement call. Good suggestion, George!

Idea: Real-world measurement of changes to Implementations vs. changes to Interfaces

October 12, 2008

An idea for a paper.

For some software project:

  • What proportion of changes are “harmless” changes to interfaces? “Harmless” interface changes: Add a function or class, remove an unused function or class, add optional arguments to a function.
  • What proportion of changes are “harmful” changes to interfaces? “Harmful” interface changes: Change function parameters, remove a function or class and references to it.
  • What proportion of changes are changes to implementations? 
  • How do these proportions change over the lifetime of the project?
The numerous implications are left as an exercise for the reader.
(I always wanted to leave something as an exercise for the reader.)

Notes on “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development”

October 12, 2008

An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development: Atul Gupta, Pankaj Jalote.

The Gist: The authors compared test-driven development with traditional development. The programmers were students and the project was a 200 line program. Conclusions: TDD leads to greater efficiency. Writing more test cases correlates with higher code quality. 

The Good: This was a straightforward study and a well-argued paper.

The Bad: I don’t believe the conclusions.

  • The TDD group was not allowed to do any design. At the end, they did not feel confident in the design that emerged. This suggests a threat to the external validity due to the small size of the project.
  • The traditional-methods group performed some manual testing. This suggests a confound due to the difference between manual/automatic testing instead of before/after testing. 
  • Their conclusions are based on ad-hoc definitions of code quality, development effort, and developer productivity. I believe that these measures must be independently validated before they are used in conclusions. For example, I reject their measure of developer productivity which relies on number of lines of code delivered.
The survey of related research highlights seven other studies, of which three found some benefit for TDD, three found no change, and one found an improvement in quality at cost of increased development time.
How I’d change the study: 
  • I’d give the groups equivalent instructions about every aspect of the software process except the testing. Both groups would be instructed to create automated tests only. The only difference would be: “Write tests before code” vs. “Write tests after code.”
  • We expect up to 100:1 differences in productivity between programmers with a non-normal distribution of skill. We don’t really know the underlying distribution. This generates a very strong expected discrepancy between random small groups. This might be one reason it’s hard to find statistically significant effects with group sizes like 8 to 10. I’d boost the sample sizes to increase the chances of finding a good effect.
My (mostly unsubstantiated) hypotheses about TDD:
  • In problem domains where the design is obvious, TDD helps.
  • TDD brings no benefit to beginners, and small benefits with a little practice.
  • TDD hurts exploratory programming.
  • TDD hurts when iterating on interfaces. TDD helps when iterating on implementations.
  • TDD encourages more thorough testing than test-after.
  • TDD creates a small increase in total development time because more tests are written.

Notes on Chapter 5 of the proceedings of the NATO Software Engineering Conference 1968

October 12, 2008

NATO Software Engineering Conference 1968, chapter 5.

This isn’t a paper like the others. It is transcription of key dialogue that took place at the NATO Software Engineering Conference in 1968. There are some famous names in the mix, like Dijkstra and Perlis.

It’s an intriguing read: 40 years ago, they wrestled with many of the same issues we have today. They talk about difficulties in software estimation, motivating and measuring programmers, measuring cost, managing large teams, communication within teams and measuring performance. I guess not much has changed.

Notes on “Unfolding Abstract Datatypes”

October 12, 2008

Unfolding Abstract Datatypes: Jeremy Gibbons

I read the first few pages, then skimmed the rest.

The Gist (I think): unfold is the key to making proofs about abstract datatypes. In the past, functional programmers have avoided abstract datatypes due to the difficulty of reasoning about them and the power of pattern matching. Gibbons gives a proof technique that eliminates the proof barrier.

 

(Why, Zachary, why?)

Three Thoughts on Developer Tools

October 12, 2008
The first two ideas are meant for developers trying to understand a code base they haven’t seen before.
  1. Run a program. Keep track of which lines of code get hit and count how many times. In the source code editor, colour by use. Help the developer focus on the important stuff. (Inspired by the performance profiling line-by-line heat map I once saw in XCode.) Advanced version: Save this information and link to test cases, so it’s easy to look at the code that a test case hits. 
  2. Run a program. For variables that get used in control statements, keep track of the domain of those variables during program execution. This would make it easy to catch some fencepost errors and also see why control statements ran the way they did, without stepping through in the debugger. 
  3. Novice programmers test their programs by running them again and again and typing in the input again and again. They use print statements to debug. Automated tests are better and not much more work. We need a way to turn a program execution with typed-in input into a high-quality automated test cases. Macros don’t cut it. This is hard but solving it is valuable. (This one has been done, but as far as I know, not yet done well.)