Back to Questions
3.7k
views4
comments
How To Translate WebResponse(Stream) To MemoryStream
f#

I've been struggling with this for a few days but I can't seem to manage this trick. I want to convert a WebResponse to a MemoryStream. I found this C# example code

1
2
3
4
5
6
7
8
9

MemoryStream m = new MemoryStream();
Stream receiveStream = myHttpWebResponse.GetResponseStream();
using (StreamReader sr = new StreamReader(receiveStream)) 
{
    while (sr.Peek() >= 0) 
    {
                  memStream.WriteByte((byte)sr.Read());
    }
}

And I've tried to translate it to F# but the results I've gotten so far--well I'm a little embarassed to post them because I can't even get the syntax right. There are two issues I keep running up against: the fact that the StreamReader.Read has to keep the state of the stream because each call to Read moves the pointer in the Stream. There's that and the fact that the C# code mutates the MemoryStream.

This is what I've come up with so far

1
2
3
4
5
6
7
8
9
10
11
12

(* open System.IO 

open System.Net

let wr = class.GetResponseStream() *)

let toMemoryStream (wr:Stream) =
    let sr = new StreamReader(wr)
    let ms = Array.empty<byte>
    Array.append ms (sr |> Array.filter(fun e -> e <> -1) |> Array.map (fun e-> byte e))
    new MemoryStream(ms)

The code chokes on the Array.filter but I'm not even sure I'm even going in the right general direction.

Is there some easier way to translate a WebResponse to a MemoryStream? Some constructor that I'm missing? I don't doubt there is some easier approach to this that I'm overlooking. Any suggestions or thoughts are welcome.

.

Something like this?

Version for .NET 4.0 (with Stream.CopyTo method)

1
2
3
4
5
6
7
8
9
10

open System.Net
open System.IO
let getMemoryStream (uri : string) = 
    let request = HttpWebRequest.Create(uri) :?> HttpWebRequest
    use response = request.GetResponse()
    use rs = response.GetResponseStream()
    let ms = new MemoryStream()
    rs.CopyTo(ms)
    ms.Position <- 0L
    ms

if required framework version is lesser - code remains the same plus CopyTo as extension

1
2
3
4
5
6
7
8
9

type System.IO.Stream with
    member this.CopyTo(other : Stream) = 
        let buf = Array.zeroCreate 4096
        let rec copy () = 
            let n = this.Read(buf, 0, buf.Length)
            if n <> 0 then
                other.Write(buf, 0, n)
                copy()
        copy()
By on 8/4/2010 5:55 AM ()

ISTM, that if you're copying the whole stream to EOF, then there's no reason for a stream at all - you may as well read a byte[] or string and use it directly, or at which point you could create a memstream from it easily via a standard c'tor for any API that really needs the stream - and I'd think this would be fast than a user copy()ing the stream-to-stream, though the lib could to something similar or make use of the response size to pre-allocate the memstream buffer, if available.

Also, I'm not a web/IO guru myself, but certain web requests are lazy and cause a continued read from the server, or at any rate you could read from the head of the stream before all the content is delivered, correct? - in which case you'd want to use the original stream to enable processing ASAP, or create something like a LazyMemoryStream that causes reads on the underlying stream on a background thread, blocking if a seek can't be fulfilled yet, and closing the underlying stream when the copy is done or the user disposes. (I'm not saying the OP needs such a thing - but it seems like such a thing would be useful if you need pointer/seek semantics but you want to be able to start asap. )

By on 8/4/2010 7:14 AM ()

Thanks to everyone that responded. I'm still trying to find some time to try out these suggestions but I just wanted to say thanks for the assistance.

By on 8/5/2010 9:53 AM ()

I've whipped up a bit of code that pumps the contents of one stream into another. It's a straightforward 'while not -1, pump a byte from one stream to another' code, just like the C# sample.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

open System.Text
open System.IO


let pump stream1 stream2 =
    match (box stream1),(box stream2) with
    | (:? Stream as inStream), (:? Stream as outStream) -> 
    
        let mutable curr = inStream.ReadByte()
        while curr <> -1 do
            outStream.WriteByte(curr |> byte)
            curr <- inStream.ReadByte()
    
    | _ -> failwith "Both inputs have to inherit System.IO.Stream"
    
//an example using a file stream instead of a web response stream
let test() =
    use fs = new FileStream(@"c:\temp\respons.xml", FileMode.Open)
    use ms = new MemoryStream()
    
    pump fs ms
    
    //extract contents from memorystream
    let buffer = Array.create (ms.Length |> int) 0uy
    ms.Seek(0L,SeekOrigin.Begin) |> ignore
    ms.Read(buffer, 0, buffer.Length) |> ignore
    
    printfn "Contents: %s" (Encoding.UTF8.GetString(buffer))

Disclaimer: I'm a bit of a noob when it comes to IO, so feel free to mock this code. :P

By on 8/4/2010 12:52 AM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper