Hey,

I like to note in my posts that I'm not a programmer, I'm a statistician using F#, Python, R etc in my work so be lenient with my below average programming skills ;)

I have a problem which I'm trying to solve using pipelines in F#, but I can't get it as efficient as by using an uglier method.

Basically what I'm doing is taking a dictionary, filtering out the elements which the dictionary has in common with an external dictionary and summing it all together.

The code with pipeline looks like this:

1
2
3

sum <- obj.dict
   |> Seq.filter (fun (KeyValue(key,_)) -> extObj.dict.ContainsKey(key))
   |> Seq.sumBy (fun (KeyValue(key, value)) -> <bunch of math>)

The code is intuitive, but it's much slower than the uglier:

1
2
3
4
5
6
7

for KVPair in obj.dict do
   let mutable extObjValue = 0.0
   let key = KVPair.Key
   let value = KVPair.Value

   if extObj.dict.TryGetValue(key,&extObjValue) then
      sum <- sum + <bunch of math>

The latter version is a lot faster than the pipeline version, and I suspect it may have to do with me doing a lookup in the <bunch of math> since I use values from extObj.dict there as well, and I do a lookup in the same dictionary in the filter step in the pipeline.

It should be added that the <bunch of math> includes lookups in both extObj and obj, as well as some other dictionary lookups from dictionaries within the key (which is an object).

Do you have any guesstimate what's slowing down the pipeline?
I ran a performance wizard from VS 2010 and the sumBy step is the heaviest line.

Thanks in advance