Jay R. Wren – lazy dawg evarlast
babblings of a computer loving fool
Posts that relate to Programming, Software Development, Software Engineering, Software Design or Architecture, and even maybe shell scripting (the programming part of systems administration)
I can’t wait to see all those people that I ain’t seen since the last time there was CodeMash. Ultimately it is the people that make CodeMash such an excellent event. Without the quality community that the CodeMash organizers have built it wouldn’t be the success that it is. Thanks organizers.
See you there!
Sing to the tune of “Christmas is coming”
CodeMash is coming
The talks are getting full,
Please check the schedule
For the things that rule.If you haven’t got the F# skills,
The Scala skills will do,
If you haven’t got the Scala skills,
The God bless you.
OK, that was lame, but I don’t care. I’m really looking forward to the excellent talks at CodeMash. Yes there will be many open spaces, and they will be splendid. The majority of CodeMash is still classic eyes forward sessions. Nearly all of them that I attended last year were brilliant.
I still haven’t picked which talks I am going to see. What talks are you planning to see?
Breaking the tradition of not giving developers even the most basic of data structures with which to work, Microsoft, in its magnanimous wisdom, chose to include a multimap implementation in the 3.5 version of .NET. This was the same time that it first gave developers a Set implementation too. Its amazing any developers tolerated not having these out of the box with .NET over the years, but I guess making them yourself is the norm for us old C/C++ types.
(I shouldn’t give MSFT such a hard time. Java doesn’t ship a multimap either, you are left to use Apache Commons or Google Collection Library. http://java.sun.com/docs/books/tutorial/collections/interfaces/map.html)
Not wanting to break all traditions, they screwed up on the naming. Calling it a multimap would have been too easy. After all, we don’t call our resizable arrays “vectors” do we? No, we call them Lists. But aren’t Linked Lists something else, well, yes they are, Lists aren’t Linked Lists, and Linked Lists aren’t Lists, at least not in the IList<> of the sense. Enough Ranting.
(http://java.sun.com/j2se/1.5.0/docs/api/java/util/List.html v. http://msdn.microsoft.com/en-us/library/6sh2ey19.aspx)
In the System.Core assembly in the System.Linq namespace there is an Interface named ILookup`2. There is an implementation of it called Lookup`2, but you can’t construct it. Yes, as if poor naming weren’t enough, It seems that MSFT doesn’t actually want you to use a multimap. Taking a look at return types and interface definitions, it seems this type exists only to support the Groupby LINQ implementation. Being the lazy programmer that I am, I do not want to write a multimap. The existence of the ToLookup extension method on IEnumerable, tells me that maybe I can get at one of these things.
In this example, assume I have a grocery shopping list which I’ve named mmap so that you remember it is a multimap. I want to organize (key) things in my list according to where I will store it, the fridge or the freezer.
var mmap = (
new [] {
new {Key = “Freezer”, Item = “IceCream” },
new {Key= “Freezer”, Item = “Blueberries”},
new {Key= “Freezer”, Item=”FCOJ”},
new {Key = “Fridge”, Item=”Milk”},
new {Key=”Fridge”, Item=”Yogurt”},
new {Key = “Fridge”, Item = “Eggs”}
}
).ToLookup(a=>a.Key, b=>b.Item);
foreach(var map in mmap)
{
Console.WriteLine(map.Key);
foreach(var val in map)
{
Console.WriteLine(“\t”+val);
}
}
//look’em up by index
Console.WriteLine(mmap[“Freezer”]);
//but that is just an IGrouping<>
foreach(var item in mmap[“Freezer”])
{ Console.Write(item);Console.Write(“\t”); }
Console.WriteLine();
Some things to note:
An expert LINQer might say “this is no different than group by”, and at first glance, I’d agree. There is an important difference. Under the hood it is likely that this is implemented exactly the same as group by, but ToLookup returns an ILookup`2. ILookup`2 extends IEnumerable[IGrouping`2]. The latter is what Groupby returns. This is important because the ILookup`2 interface exposes the indexer so that you can get at your map by indexing by key.
I wish I could just new up a Lookup and add items rather than jumping through hoops (see a future post). But I’m happy to never write another multimap implementation again.
I’m not just hoping, but I’m expecting to have my mind blown.
Sometimes even open spaces is too structured. The structure is good, but people feel a need to stay on topic. I appreciate this, but I also like the completely unstructured total geek talk that you can get in the hallway. Michael Letterle and Joe O’Brien blew my mind with some conversation last year. I’m sure it will be something else this year, and I can’t wait.
For what are you hoping and expecting at CodeMash? Anything mind blowing?
I’m excited to so something weird at CodeMash:
Adding things to the Open Spaces board that I know nothing about! What better way to learn about something and meet people who know those things at the same time. Maybe there will be someone (other than Dan) who knows about writing Brew applications and can tell me that I’m crazy for even thinking about it.
What are you excited to do at CodeMash?
I’m following the countdown here: http://www.isitcodemashyet.com/
I’m looking forward to learning more about Windows Azure. I know enough about Google App Engine thanks to some awesome coverage at Michipug meetings, but Azure is still nebulous to me.
What are you looking forward to learning at CodeMash?
Something I want to get out of CodeMash:
I want to ask F# experts more about Workflows. I think I grok Async Workflows, but workflows in general are still not a tool which I feel I have mastered. When would I use one? Why would I use one? What is the point again? I always forget.
What do you want to get out of CodeMash?
Jared Parsons wrote a post very similar to one that I have been wanting to do for a while. Check it out: http://blogs.msdn.com/jaredpar/archive/2008/12/02/mapping-linq-to-f.aspx
His post maps the common C#/VB.NET LINQ methods to F# function names.
I like to think about this a little differently. There are a bunch of well known functions in programming which tend to get used a bit more in functional programming. C#/VB.NET and the BCL names things rather unintuitive if you are accustomed to the functional names for these functions.
The basics:
| map | Select |
| reduce | Aggregate |
| filter | Where |
To throw a little more confusion in the mix, there are aliases and special cases for reduce.
Fold is an implementation of reduce which implies some order to how the reduction is performed. Usually fold is an alias for fold right, which means go from left to right. The other is fold_right which does the same thing but goes from right to left.
Sum, Max, Min and Average are all special cases of Aggregate or reduce.
If you are new to functional programming it will help to remember all of this. Read the wikipedia page for map: http://en.wikipedia.org/wiki/Map_(higher-order_function) and be sure to look in the See also section for the reduce page and the external links for this link: http://www.cse.unsw.edu.au/~en1000/haskell/hof.html
— begin joke here —
F# does an interesting thing and names its generic generator function “unfold” since it is the opposite of fold. This means it is the opposite of reduce. Since any specific functions implemented with fold can be considered a reduction function, and in chemistry reduction has an opposite, I like to call the concept of unfold by the name “oxidate” or “oxidize”. e.g. here is an oxidation function for Fibinacci sequence until the value is one million. It is lifted from Chris Smith’s blog:
let fibs =
(1,1) |> Seq.unfold
(fun (n0,n1) ->
if n0 <= 1000000 then
Some (n0, (n1, n0 + n1))
else
None)
Finally, there is one more similar terminology: generators.
Python calls methods like this “generators” although the docs in PEP 255 also calls them “producer functions”. PEP 289 defines a short hand expression for these things which is very similar to F# sequence expressions.
Now that IronPython is 2.0 final and we know F# will be shipped with Visual Studio 2010, please remember that sequence expressions in F# and generator expressions in python are just two forms of oxidation.
Yes, I am crazy. Why would any programmer want this combination? I think it is a combination of wanting to work with the best tools, in this case Visual Studio 2008 and CodeRush and wanting to target Win32, OSX, and Linux all at once. The very nice part is that thanks to the hard work of other people, you don’t even have to run in Mono on Win32.
[STAThread] static void Main() { Application.Init(); Window myWin = new Window("My first GTK# Application! "); myWin.Resize(200, 200); myWin.Destroyed += new EventHandler(myWin_Destroyed); Label myLabel = new Label(); myLabel.Text = "Hello World!!!!"; myWin.Add(myLabel); myWin.ShowAll(); Application.Run(); } static void myWin_Destroyed(object sender, EventArgs e) { Application.Quit(); }
Enjoy your 3rd option for a pure .NET programming GUI Toolkit on Win32. Winforms and WPF are great, but GTK# does fill a certain niche.
Its no secret that I came to C# after doing lots of Perl. I treasure my Perl experience. Many claim that Perl is unreadable, but I argue that the language is as good as the programmers using it. I’ve seen pages of equally unreadable PHP, VB, and yes, even C#. I’ve used Perl libraries as references for how things work. Many Perl libraries are examples of great highly readable code. Language doesn’t matter.
Perl’s documentation is a testament to open source languages. I learned Perl with perldoc installed an the perl man page. It taught me the Perl way of data structures and as I got more advanced I ran into Arrays of Arrays(AoA), Arrays of Hashes(AoH), Hashes of Arrays(HoA) and Hash of Hashes(HoH). These are all defined in the perllol perldoc.
Some of the most confusing Perl that I ever did read or write was confusing because instead of building explicitly named types, convention was relied on too much over configuration and everything was just a HoHoHoHoHoHoH. I think it was a joke about Santa Clause.
All that said, there are times when you really want a HoH or a HoHoH or a HoHoHoH. That is Dictionary<string,Dictionary<string,string>> or Dictionary<string,Dictionary<string,Dictionary<string,string>>> or Dictionary<string,Dictionary<string,Dictionary<string,Dictionary<string,string>>>> for you non perl types.
A little over a year ago, Eilon Lipton wrote a post on using anonymous types as Dictionaries. I rather liked it, but That was just a H (Dictionary<string,string>). I recently had the need for a HoHoA.
I’m not advocating using this code or ever writing code like this. In fact, I discourage it. If you can find a way to do the same thing but with compile time checks, please do that instead. The problem with using anonymous types as dictionary literals is that you won’t know until runtime that you have made a mistake. WRITE YOUR UNIT TESTS!
public static Dictionary<string, string> ToDictionary(this object source) { var dict = new Dictionary<string, string>(); var props = source.GetType().GetProperties(); Array.ForEach(props, p => dict.Add(p.Name, (string)p.GetValue(source, null))); return dict; } public static Dictionary<string, IList<string>> ToDictionaryOfStrings(this object source) { var dict = new Dictionary<string, IList<string>>(); var props = source.GetType().GetProperties(); Array.ForEach(props, p => dict.Add(p.Name, (IList<string>)p.GetValue(source, null))); return dict; } public static Dictionary<string, Dictionary<string,string>> ToDictionaryOfDictionaries(this object source) { var dict = new Dictionary<string, Dictionary<string, string>>(); var props = source.GetType().GetProperties(); Array.ForEach(props, p => dict.Add(p.Name, (Dictionary<string,string>)p.GetValue(source,null).ToDictionary())); return dict; } public static Dictionary<string, Dictionary<string, IList<string>>> ToDictionaryOfDictionariesOfStrings(this object source) { var dict = new Dictionary<string, Dictionary<string, IList<string>>>(); var props = source.GetType().GetProperties(); Array.ForEach(props, p => dict.Add(p.Name, (Dictionary<string, IList<string>>)p.GetValue(source, null).ToDictionaryOfStrings())); return dict; }
Yes, you have to call the right method, it isn’t magic. Yes, these extension methods pollute all objects with their clutter. They do work when you need them though!
var things = new { Humans = new { Johny = new[] { "Guitar", "Docks" }, Gina = new[] { "Diner", "Pay" } }, Dogs = new { Sparky = new[] { "beer", "baseball" }, Toto = new[] { "witch", "kansas" } }, Marsians = new { Marvin = new[] { "death ray", "moon laser" }, Quato = new[] { "speech", "power" } } }.ToDictionaryOfDictionariesOfStrings(); Assert.IsTrue(things.ContainsKey("Humans")); Assert.IsTrue(things.ContainsKey("Dogs")); Assert.IsTrue(things.ContainsKey("Marsians")); var humans = things["Humans"]; Assert.IsTrue(humans.ContainsKey("Johny")); Assert.IsTrue(humans.ContainsKey("Gina")); var gina = humans["Gina"]; Assert.IsTrue(gina.Contains("Diner")); Assert.IsTrue(gina.Contains("Pay")); var dogs = things["Dogs"]; Assert.IsTrue(dogs.ContainsKey("Sparky")); Assert.IsTrue(dogs.ContainsKey("Toto")); var sparky = dogs["Sparky"]; Assert.IsTrue(sparky.Contains("beer")); Assert.IsTrue(sparky.Contains("baseball"));