Back to Questions
4.0k
views4
comments
How to handle Exceptions in Async Parallel code
f#webparallelasynchtml

Im trying to get the below code to work but it cant handle exceptions in the httpAsync function . I have tried using try with but it keeps saying you need to return string instead of uint ( when im returning in) , have also tried Async.Catch

Any help would be appreciated

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

open System.Net
open System
open System.IO
open System.Text.RegularExpressions
open Microsoft.FSharp.Control.WebExtensions
open Microsoft.FSharp.Control

let httpAsync (<A href="string">url:string</A>) = 
    async { 
         //    try
                let req = WebRequest.Create(url)    
                let! resp = req.AsyncGetResponse()  //Todo get errors and timeout.

                let stream = resp.GetResponseStream()
                let reader = new StreamReader(stream)
  
                let contents = reader.ReadToEnd()      // Todo make Async           let! contents = reader.AsyncReadToEnd()
                return contents  // return (url, html)
    
        }  

let parallelWorker n f =
    MailboxProcessor.Start( fun inbox ->
        let workers = 
            Array.init n ( fun i -> MailboxProcessor.Start(f))
        let rec loop i = async {
            let! msg = inbox.Receive()
            workers.[i].Post(msg)
            return! loop ((i+1) %n)
        }
        loop 0
    )


let linkPat = "href=\s*\"[^\"h]*(<A href='http://[^&\"]*)\'>http://[^&\"]*)\</A>""
let getLinks (txt:string) =
    [ for m in Regex.Matches(txt,linkPat) -> m.Groups.Item(1).Value ]


let collectLinks (<A href="string">url:string</A>) =
    async { // An Async web request with a global gate

        let! html =  httpAsync(url) 
        let links = getLinks html

        do printfn "finished reading %s, got %d links" 
                url (List.length links)

        return links }

let urlCollector  (parallelcount:int) = 
    parallelWorker parallelcount (fun self -> 
        let rec waitForUrl (visited : Set<string>) =
            async { if visited.Count < 10 then
                         let! url = self.Receive()
                         if not (visited.Contains(url)) then
                              Async.Start 
                                (async { let! links = collectLinks url
                                         for link in links do
                                             do self <-- link })
                         return! waitForUrl(visited.Add(url)) }

        waitForUrl(Set.empty))

urlCollector(8) <-- "<A href="http://news.google.com">http://news.google.com</A>"
System.Console.Read() |> ignore
.

See code below for some pointers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

 

open System.IO
open System.Net

let httpAsync (url : string) = 
    async { 
        try
            let req = WebRequest.Create(url)    
            let! resp = req.AsyncGetResponse()

            let stream = resp.GetResponseStream()
            let reader = new StreamReader(stream)

            let contents = reader.ReadToEnd()
            return contents
        with e ->
            // when successful, the 'try' block returns an Async<string>,
            // so when exception that we catch, must also return 
            // Async<string>
            return sprintf "An Exception happened: %s" e.Message 
    }  

// Another common idiom is to have the function named with a 'try' prefix
// and return an option (return Some(result) for success, and None for
// failure)
let tryHttpAsync (url : string) = 
    async { 
        try
            let req = WebRequest.Create(url)    
            let! resp = req.AsyncGetResponse()

            let stream = resp.GetResponseStream()
            let reader = new StreamReader(stream)

            let contents = reader.ReadToEnd()
            return Some(contents)
        with e ->
            return None
    }

Note that none of this is really related to 'async', this is all about how to handle exceptions in general, and it applies equally well to async.

By on 11/29/2009 9:42 AM ()

Thanks very much...

New language ,feel like an idiot.

Anyway I tried these options before 

1
2
3
4

   return contents
  with 
////                //| _ -> failwith  "link failed" 
////                e -> printfn "%s" e.Message; "link failed" // |> ignore

but theses didnt work .. yet your similar code does . I was trying to return the string link failed do you have to have an explcit return ?

Also the code i pasted works as a script (.fsx) but fails as code (.fs)

I get this error 

1

Error 1 Method or object constructor 'op_LessMinusMinus' not found C:\Users\Ben\Documents\Visual Studio 10\Projects\ConsoleApplication1\ConsoleApplication1\Program.fs 102 17 ConsoleApplication1
By on 11/29/2009 3:50 PM ()

The reason you need "return" there is because you're in a "computation expression" or "workflow" (inside the async { } part). When inside the async part, you need to return an 'a Async, not just any type such as a string. So you need to go from string to string Async. The special return keyword, available only inside comp-exprs, wraps your expression into the proper return type.

If a workflow ends with just some expression, the compiler tacks on a call to that workflow's "Zero" function, which returns a default/zero value. In the case of async, this is unit Async. So, when you have an async workflow that ends with a string, it's as if you wrote "someString; return ()". You can see that here:

1
2
3
4
5
6

> let x = async { "test" };;
let x = async { "test" };;
----------------^^^^^^
stdin(3,17): warning FS0020: This expression should have type 'unit', but has type 'string'. Use 'ignore' to discard the result of the expression, or 'let' to bind the result to a name.

val x : Async<unit>
By on 11/29/2009 5:04 PM ()

ty all

By on 11/29/2009 6:17 PM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper