db4oProviders Release 1.0

db4oProviders Release 1.0 now available.

Description: Custom ASP.NET 2.0 providers which use db4o as back-end. Beginning with Membership Provider, Role Provider, and Profile Provider. Code is based on MSDN sample code and is unit-tested. Version 1.0 uses db4o 7.4 and C# 3.0. Prior versions of source use db4o version 6.1 and C# 2.0.

http://www.codeplex.com/db4oProviders

The ascendency of Lord British

Richard Garriott was one of my heroes when I was 15. He wrote the Ultima series of computer games, and the one I owned, Ultima IV, was in a class of its own for making morality a central aspect of gameplay. Your character's moral status was a measurement of your compliance, throughout the game, with eight virtues. I still have the game box, including a super-cool cloth map of "Britannia." And I can still picture the pages of a magazine interview of Richard back then. His words which I dwelled upon amounted to: if you want to develop cool software, the most important thing is to master the dry technicalities of programming. That interview was very inspiring to me.

Now, Richard, the son of a NASA astronaut, is spending his own fortune on a ten-day spaceflight. When he looks out the window back at Earth, might he see something like this?

Study Groups for Objectivists

SGO, open to all Objectivists, has completed its first study group, covering chapter 5 of Introduction to Objectivist Epistemology. It was a great success with six contributing members, and I gained clarity on some points in the chapter.

I developed the SGO software from scratch using C# and db4o. I like to have complete ownership of an implementation like this, because it lets me experiment with improved architectural elements -- such as using an object database (db4o) rather than a relational database, and trying my hand at Domain Driven Design.

Two study groups are slated for the rest of this year. I'll be moderating one covering the first half of Ayn Rand's lecture and essay, "The Objectivist Ethics."

Here's my final post in the study group we just completed:

The challenge question I assigned myself a few weeks ago was this: Most people would think that knowing a word's definition is more than you need to know to understand a word, i.e., that one doesn't need to be able to formally define "justice" in order to understand what it is. How do we know that this is wrong?

One theme of chapter 5 is that knowledge is contextual: every element of one's knowledge is built upon and supported by other elements of one's knowledge, all the way down to one's actual perceptual experiences. Chapter 3 showed one way in which knowledge is contextual: conceptual hierarchy. For example, the concept "furniture" cannot be grasped before grasping concepts such as "table" and "chair" -- because tables and chairs are not similar enough in immediate perception to warrant "furniture" as a first-level concept; but "table" and "chair" as concepts are similar with regard to the more abstract distinguishing characteristic of "furniture": tables and chairs can support the human body and/or other objects. So, "table" and "chair" (or similar) are part of the cognitive context which is necessary to grasp "furniture". See 3.12 (chapter 3, paragraph 12).

The principle that knowledge is contextual has much broader application than just conceptual hierarchy. She gives an example at 3.13: "habitation" is not a unit of "furniture", but it is part of the necessary context for grasping "furniture" as an adult.

How is the contextual knowledge which supports a concept specified? By defining its units. A definition makes explicit the concepts and relationships one must grasp to distinguish a concept's units. When there is some question as to exactly what the essential characteristic of a concept is, and one just "kinda" feels what he or she means by a word, the concept is not actually retained because it has become unhinged from those other pieces of knowledge needed to identify its units.

Given the nature of concepts, no concept can exist on its own in one's mind, it must be properly related to the rest of one's knowledge -- which means in practice that it must be immediately definable. Otherwise it is at most an approximation of knowledge.

From C# to F#, Part 1: Expressions with side-effects

I'm going to start a series of posts as I learn F#, working through the excellent book Expert F#, which was published last month (and I wholeheartedly recommend it over Fundamentals of F#, by the way). The premise of these posts is that the reader and I have advanced C# skills, but we're new to F# and functional programming. The caveat is that since I'm new to F#, I'll make mistakes, but I'll come back and fix them as soon as I know better.

What is F#? From the download page for version 1.9.3.7:

F# is a variant of the ML programming language for .NET and has a core language similar to OCaml. F# is a mixed functional/imperative/object-oriented programming language excellent for medium/advanced programmers and for teaching. It also can be used to access hundreds of .NET libraries, and the F# code can be accessed from C# and other .NET languages.

Functional Expressions

F# gets its name and notoriety from the fact that it is, among other things, a functional language, so let's start off with functional code. Within an fsi.exe interpreter session, I'm typing the blue:

> let add x y = x + y;;

val add : int -> int -> int

Everything in blue up to the semicolons is an expression which is evaluated. The semicolons just tell the interpreter to go ahead and evaluate everything typed up to that point, I won't be continuing the expression on the next line. Here the let expression evaluates to a function, add, which maps two integers to their sum.

Next, the line of feedback from the interpreter displays the results of the evaluation -- hey, we've created a value, named add, that is a function of a certain type. The type of the function looks odd at first, but let's keep moving and try it out:

> add 3 4;;
val it : int = 7

The expression add 3 4 evaluates to an integer value of 7. The name it is just a placeholder for the unnamed value (7) that resulted from the evaluation.

Have you noticed that the type int was assumed by the interpreter for x and y? That's what F# does, it infers the types of an expression based on what the expression does and how it does it. While it may seem arbitrary that x and y are inferred as integers here rather than as floats or whatnot, the rules of type inference are well-defined (this one is directly inherited from OCaml, F#'s daddy). Type inference has some serious selling points: values are strongly typed without having to be explicitly typed. In some cases, when a type cannot be unambiguously inferred, F# requires an explicit type specifier -- this is the price for having the best of both worlds.

Now why is the type of add int -> int -> int? This means that add is a function which maps an integer to a function which maps an integer to an integer. That's a mouthful, but what does it mean? Don't try this in C#:

> add 1;;
val it : (int -> int) = (fun:it@11)

Look, no error! We can apply just one parameter (x) to add, and that expression (everything in blue) evaluates to a function which maps an integer to an integer. Don Syme, F#'s creator, calls such a result a residual function. Let's package this up in a named function, inc, that we can use later:

> let inc = add 1;;

val inc : (int -> int)

> inc 4;;
val it : int = 5

So inc is a function which maps one integer to another -- by evaluating, in effect, add 1 y. Now we can better understand the type of add:

int -> int -> int

This is the type of a function which maps one integer (the left int) to a function of (on the right) type int -> int, that is, to a function which, like inc, maps one integer to another.

...With Side-Effects

F# is not a pure functional language, it is also a fully-featured imperative and OO language. I'm led to believe F# could even be used as a total replacement for C# without ever using it's functional aspects. Here's an expression:

> printfn "Hello!";;
Hello!
val it : unit = ()

First, note that this is an expression, not a statement -- there are no statements in F#, and this is a clue as to why some things do not work as the C# brain expects. When this expression is evaluated, it results in a side-effect, the printing of Hello! to the console. The type of the expression is, we are told, unit, which is just F# for void. Such imperative code can show up in a function too. Let's try this:

> let hello = printfn "Hello!";;

val hello : unit

Hello!

Oops! I hoped to be defining a parameterless function, but it turns out hello is not a function, we can see this from its type, the hapless unit. Interestingly, the side-effect of printing Hello! to the console already occurred during the evaluation of the let expression.

A commentor has kindly pointed out that to define a function that takes no parameters, we can indicate one "empty tuple" parameter:

> let hola () = printfn "Hola!";;

val hola : unit -> unit

> hola ();;
Hola!

This is what I was after, a function which maps unit to unit but has a side-effect. The empty parens () is not an empty parameter list, that's just what it looks like to the C-derived-language eye. It is actually a special parameter, the empty tuple. I'll discuss tuples in part 2.

Mixed

Let's define a more interesting function with side-effects:

> let addandtell x y = printfn "The number is %d" (add x y);;

val addandtell : int -> int -> unit

> addandtell 8 9;;
The number is 17
val it : unit = ()

Function addandtell maps from two integers to unit, and it has the side-effect of printing a message with their sum out to the console. This time the side-effect from printfn happens exactly when sub-expression add x y evaluates to an int -- which happens when the 8 and 9 are supplied to addandtell. If we were to next supply addandtell with 10 and 11, then add x y would re-evaluate, causing the printfn sub-expression to re-evaluate and have a new side-effect: printing The number is 21 to the console.

As we did earlier with add, we can apply just one parameter to addandtell, and then use the residual function.

> let incandtell = addandtell 1;;

val incandtell : int -> unit

The sub-expression addandtell 1 is a residual function with a side-effect. The side-effect won't occur until the add "inside" of addandtell gets all of the parameters it is waiting for and can evaluate to the integer which printfn takes. Let's make it happen:

> incandtell 13;;
The number is 14
val it : unit = ()

And there's the side-effect, the printout of The number is 14.

Don't take the "waiting for" metaphor here literally. There's no blocked thread here. A residual function is produced by supplying some-but-not-all of the parameters to a function, so in a logical sense the original function is "waiting for" the rest of its parameters.

Next

In part 2 I'll look at some of F#'s special data types, and do some real work with recursive functions.

Out, out damn carpet wrinkle!

I've been skeptical of LINQ to SQL from the beginning. There is quite a bit of history (to understate) of attempts to "solve" the problem of the impedance mismatch between object-oriented applications and relational databases, and, like pushing down a carpet wrinkle, the problem never seems to go away, it just pops up somewhere else.

Now that the evidence is starting to come in, I can stop nursing my irresponsible speculation. And the web is confirming my suspicions:

1 - Scott Guthrie's most excellent introduction to LINQ to SQL is not one long post, nor three or even five long posts, but NINE long posts. Cancel my meetings, I've got a sharper axe!

2 - The stuff hasn't shipped and here's a very nice "gotchas" post. Turns out you have to know what's happening behind the scenes or your pleasant-looking C# will translate into horrendous SQL, after all.

3 - There seem to be about 3000 sessions on LINQ and the Entity Framework at this year's Dev Connections conference in Vegas.

Not to say LINQ to SQL isn't perhaps an improvement over classic (tempus fugit!) ADO.NET for many situations. But when I see a new subculture of support forming for an upcoming technology, I may just wonder if the previous stuff I'm used too isn't so bad.

To my mind, even more exciting than expressing queries in C# so that at run-time they can be automagically mapped to SQL queries, is expressing queries in C# that don't get mapped or translated at all, they just are what they are and do what they say.

db4o ASP.NET Providers

My open-source project of db4objects-backed Membership, Role, and Profile ASP.NET Providers is up on CodePlex here.

db4objects

db4objects is an object database that I'm fairly enamored with these days. Whereas with a relational database I'd have to write lots of SQL and plumbing to read and write objects, with db4objects in .NET 2.0 I can store an object by calling Set(object), and load an object by calling the Query<>() method with an anonymous delegate that simply expresses the identity of the target, like this:

IList<EnrolledUser> results =

db.Query<EnrolledUser>
(

delegate(EnrolledUser u)
{

return u.Username == username &&
u.ApplicationName ==
this.applicationName;

}

);

The Achilles' heel (to be dramatic) of db4objects for .NET is that it requires Full Security CAS privileges, since it uses reflection and Win32. That's a problem for hosted web applications which are constrained to the Medium Security model -- which is very common and proper. I've thought 'round and 'round this problem, and there's no readily plausible solution other than finding a host which is willing to run your web app with Full Security -- which is, of course, in general a very bad idea!

The Providers

My project contains implementations for three core ASP.NET providers so far. They are based on MSDN sample code, and they are "well" unit-tested, meaning: I tried to retroactively "drive" most of the behavior with tests, but there are some holes.

There was an older, similar project on SourceForge here, but that one was not based on the MSDN sample code, was not unit-tested, hadn't been touched in 18 months, did not include a Profile provider, and used an older version of db4objects and the older method of querying. So I wrote this one. And let me tell you, it is not easy to unit test a Membership Provider; I never could have done it had I not found an archived post of a blog that doesn't exist anymore which explained how to hack it!

I'm curious myself about what bugs I'll find once I (or others) start using this project more. It's been downloaded more than 40 times already, so I hope -- and fear -- some bug reports sometime soon.

Index cards as bricks

There is an interesting post by Charles Petzold on how and why he wrote Applications = Code + Markup the way he did. The most interesting thing to me is the peek into his writing process:

I spend a lot of time and energy organizing the material in my books. I draw big dependency diagrams on white boards; I make index cards for various topics and then lay them out on the floor in various experimental orders; I write sketchy early versions of half a dozen chapters to test if the material flows correctly. The ordering of material is a central problem in writing a programming tutorial, and I struggle to get it right. I want each chapter to progressively build on the last, like bricks in a wall of knowledge.

Good books always read as if the author sat down with perfect knowledge of the topic and then just said it on paper. Who knew authors litter their office floors with index cards, and their garbage cans with sketchy early versions of chapters. All in a quest to solve a problem that, ironically, the author doesn't have: the reader's lack of knowledge and context.

A large part of the difficulty in building the reader's "wall of knowledge" is a problem of getting the hierarchy right. There must be a buildup of logically dependent pieces -- the epistemological equivalent of a game of Tetris. Some pieces are especially crucial to the goal of completing the wall:

In any book I write, one of my constant goals is to demystify the [topic]. Anything that seems like "magic" is a target for me.

For "[topic]", he actually said "API", but I like this as more general advice for a writer: to maximize the value of your book, make accessible that which was previously least accessible. A corollary I'd like to note: if the topics are already sufficiently accessible, then a new book isn't worth writing.

Petzold regrets the lack of screen shots in ACM, but I have to wonder how much longer the book would have been with them. As it is, I'm on page 55 of 976.

Scratch

Perhaps the best first programming language: Scratch.

Sample code:





When I was in college, a professor was hired whose research area was graphical programming languages, and I remember her presentation looked like this.

Functional programming versus patterns

I haven't worked with a functional programming language since college, where LISP was big; I haven't touched Ruby or the C# 3.0 preview as of yet. Today I learned a couple of fascinating things about them: functional programs inherently support concurrent execution (Hejlsberg articulates this); and -- more of a "wow!" for me -- functional languages don't need no stinking design patterns:

In a functional language one does not need design patterns because the language is likely so high level, you end up programming in concepts that eliminate design patterns all together. Once such pattern is an Adapter pattern (how is it different from Facade again? Sounds like somebody needed to fill more pages to satisfy their contract). It is eliminated once a language supports a technique called currying.

I admit appreciating the dig at the pretentious and unreadable Gang of Four. But the important point is this: although we software engineers think of them as these happy helpful things, the very utility of design patterns reflects the inherent impedance between (imperative) programming and the software solutions we want to build. There is tremendous conceptual baggage involved in implementing a working system, and much of that baggage is necessitated by our implementation paradigms.

Now, whether or not this doesn't have to be the case, and functional programming's poop doesn't smell, as coffeemug at defmacro.org is implying, I don't know.

Is CCDD part of TDD?

Does test-driven development (TDD) entail what I'd like to call client-code driven development (CCDD)? Sells contrasted them in 2004, while admitting that what he called "client-driven development" (confusingly, since it more readily labels something else) may just be a degenerate case of TDD. Yesterday, Box noted what a good thing Sells's approach is. I agree.

Although CCDD -- which, in ad hoc form, must be just about as old as programming itself -- doesn't seem to be referenced explicitly in some common overviews of TDD, I find that in practice TDD entails CCDD.

In TDD, the software engineer follows this recipe for success:

1) write test A for a requirement
2) run test A, see it fail
3) implement production code
4) run test A, see it pass
5) go to 1, proceeding to test B

This appears simpliciter to define a fail-safe process for writing code. But if this is all you've got, as soon as you start, big questions arise about how to proceed that this recipe does not address. The rub is that step #1 assumes a test of something specific -- but what? Oftentimes, for me, the target production code is some method, but how was it chosen? The tests for method Foo() may drive the implementation of Foo(), but they cannot drive the need for Foo().

A wider scope is needed to choose which Foo() and which Bar() to create when and where. In a sense, one can stay true to the TDD dogma and say that higher-level tests -- ultimately, the user tests for stories and requirements -- have the necessary scope to drive all narrower decisions. My point is that, in practice, the meat of this process entails writing, over and over again, client-code calls to code that doesn't exist yet -- which forks a sub-process for the software engineer: these tendered client-code calls are the proximate causes of dependent units of code, whose implementations will be driven by their own tests.

In effect, tests will drive an implementation, which itself often "drives" the need for more code and thus more tests. CCDD is an intrinsic part of TDD.

WPF/E

CTP of WPF/E today. Utter nonsense, you say? Well, I say it's a glimpse into the post-HTML apocalypse -- and a beautiful thing it will be!