Back to Questions
1.8k
views9
comments
How to avoid a stack overflow when receiving MailboxProcessor messages?
f#async

So, I've been playing with MailboxProcessors since they're relatively easy for me to wrap my head around. I ran into a Stack Overflow, though, when I tried to send a bunch of messages to one particular MailboxProcessor. Is there any way to use these things for more than 250 or so messages?

It seems to happen for even the most basic code:

1
2
3
4
5
6
7
8
9
10
11
12
13

#light

open Microsoft.FSharp.Control.Mailboxes
let logger = MailboxProcessor.Start(fun inbox ->
    let rec run i = 
        async { let! s = inbox.Receive()
                do printfn "%s" s
                return! run (i + 1) }
    run 0)

for i = 0 to 500 do
    logger.Post("Message!")
System.Threading.Thread.Sleep(10000)
.

To avoid stack overflow exceptions mail boxes need to be tail recursive (i.e. the last thing they do is call the recursive function). The compiler will spot the tail recursion and turn it into a loop to avoid stack overflow problems.

The mail box your posted is tail recursive so can handle the 500 messages you send it (and many more), with out a problem. If you post the code of the failing mailbox then I'll be happy to take a look and try and figure out why it is no longer tail recursive.

Cheers,
Rob

By on 4/1/2008 8:32 AM ()

Huh, oddly enough it works in FSI, but not when I compile it. Any idea why that could be? I filed a bug about it, but I'd like to get it functional. :)

By on 4/1/2008 8:42 AM ()

It's working when its compiled on my WinXP 32 bit machine at work. I have version 1.9.3.7 installed, not sure if this is the latest. I have vista 32 at home I'll try it on that too when I get back.

By on 4/1/2008 8:49 AM ()

I haven't yet looked into the details, but the program does overflow the stack on Windows XP x64 quite quickly. There is no overflow if one forces the program to run on the 32-bit CLR by using the compiler option "--platform x86". It's quite possible that this is a bug in the tail call optimization of the 64-bit CLR since the implementation of the tail call optimization significantly differs between the 32-bit and 64-bit CLRs. If this is indeed a bug (or "implementation limitation") in the CLR, that would be quite bad, because it could take ages before it gets fixed (and even longer until the fix gets deployed).

Edit: See this page for an unofficial list of conditions that prevent the 64-bit JIT(s) from generating a tail call.

By on 4/1/2008 10:13 AM ()

Well, at least I can force x86 mode and have a working app. [:)]

By on 4/1/2008 10:33 AM ()

Hi all,

Yes, this looks like a 64-bit tailcall issue. In the next release we are planning to address this in the FSharp.Control implementation (by hijacking the asynchronous continuation onto a new work item every so often). For now forcing 32-bit is your best workaround.

thanks

don

By on 4/3/2008 1:23 AM ()

In case you've investigated this issue further, it would be interesting to hear what exactly the problem here was, so that one could try to avoid it in the future.

BTW: It would be really good if the .net JIT team published some kind of specification and made guarantees with regard to tail calls. Without guarantees tail call optimizations are pretty useless.

Stephan

By on 4/3/2008 5:40 AM ()

I've tried to use the countingAgent as well (modified slightly to function as a progress reporting agent). Here's the agent:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

 

type internal msg = Increment of float | Fetch of AsyncReplyChannel<float> | Stop
type ProgressAgent() =
   let progTrigger, progEvt = Event.create()

   let proc = MailboxProcessor.Start(fun inbox ->
      let rec loop(n) =
         async { let! msg = inbox.Receive()
                     match msg with
                     | Increment m ->
                        if int(n+m) > int(n) then progTrigger(int(n+m))
                        return! loop(n+m)
                     | Stop ->
                        return ()
                     | Fetch replyChannel ->
                        do replyChannel.Reply(n)
                        return! loop(n) }

      loop(0.0))

   member this.ProgressChangedEvent = progEvt
   member this.Increment(n) = proc.Post(Increment(n))
   member this.Stop() = proc.Post(Stop)
   member this.Fetch() = proc.PostAndReply(fun replyChan -> Fetch(replyChan))

I seem to be having the same problem as Stephan. After a few thousand calls to progressAgent.Increment(n) it just dies (in my "production" code I have a trace function that outputs progress updates, and those trace outputs stop). I also wondered if there could be a problem with the agent in a parallel environment, but in FSI and using the parallel extensions described here, I was able to do the following with no problems:

1
2

> let agent = new ProgressAgent()
> pseq [1..1000000] |> PSeq.fold (fun (u:unit) item -> agent.Increment(1.0/1000000.0)) ()
By on 8/2/2009 2:42 PM ()

The one I posted is actually failing for me with F# 1.9.3.14 on Vista 64. Does it work for you?

By on 4/1/2008 8:37 AM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper