Back to Questions
2.5k
views5
comments
Need help on parsing binary data
f#parsing

Dear All:
I am a newbie for F#, now I need some help from you.
I have to parse some data in binary format from internet. To make things easier, I saved the binary data into a binary file in my PC.
The binary data has the following format:

1
2
3
4
5
6
7
8
9

++++++++++++++++++++++++++++
Type    Value(Hex)
---------------------------------------------------
Byte     {4E}    
Short    {00 01}    
String    {31}    
Integer  {41 7C 00 00}<
Long     {00 00 00 00 01 0F 9F 75}
++++++++++++++++++++++++++++

The data have other binary information with the similar structure but repeated from 1 up to 40 times,
I want to parse the binary data and get the value and covert the number to be represented in decimal number, and get the string value to ASCII.
For example, in the data provided as the explanation, integer {41 7C 00 00} is
15.75 in decimal and the long hex value {00 00 00 00 01 0F 9F 75} is 17,801,077 (I use "," is just to make the number to be easily read, the "," can be deleted); the string {31} is "1" in ASCII.
Since I do not quite understand FParsec, besides, my binary data structure seems to be rather simple, it has only 4, 5 different types, so I do not want to use FParsec.
However, since I am a newbie for F#, I want to know if anyone can offer some sample code to parse the data and convert the values.
Any help is greatly appreciated!
Thanks,
John

.

This doesn't have to be specific to F# - you can just look up reading files in C# and get almost all the info you need.

In summary, you'll want to use the .NET classes System.IO.File to read the file into an array or stream, then use System.BitConverter to convert numbers and System.Text.Encoding to read text.

You can extract parts of an array using F# slicing syntax. For example to grab the first 3 bytes of an array:

1
2
3
4
5
6
7
8
9
10

 

> let input = [|0x0auy;0x0buy;0x0cuy;0x0duy|];;

val input : byte [] = [|10uy; 11uy; 12uy; 13uy|]

> let arr1 = input.[0..2];;

val arr1 : byte [] = [|10uy; 11uy; 12uy|]

The encoding you posted is big-endian, so you'll want to reverse the byte arrays before you call the conversion methods (use Array.Reverse). For example, "Integer {41 73 00 00}" is a 32-bit float (called "Single" in .NET for some reason). So you'd do:

1
2
3
4
5
6

 

> System.BitConverter.ToSingle([|0x00uy; 0x00uy; 0x7Cuy; 0x41uy|], 0);;

val it : float32 = 15.75f

The text, for example, can be converted like this:

1
2
3
4
5
6

 

> System.Text.Encoding.UTF8.GetString([|0x31uy|]);;

val it : string = "1"
By on 11/22/2009 5:13 PM ()

Hello, MichaelGG:
First, thank you very much for your quick reply.
I know I can use other .NET language, but I have done most part of my program already in F#, so I do not want only this small part in other language.
Besides, if you read my post again, you will find that the similar data structure repeats from 1 to up to 40 times; so I want to know using F#, is there any way I can re-use the code to repeat the same process for 1 to 40 times?
Thank you for you, if you have a good idea for re-using the code/function, please let me know.
Finally, since the binary data I put here is taken from internet, using the following function:

1
2
3
4
5
6
7
8
9

+++++++++++++++++++++++++++++
let GetWeb(url:string) =
    let req = WebRequest.Create(url)
    let rsp = req.GetResponse()
    use stream = rsp.GetResponseStream()
    use reader = new StreamReader(stream)
    let GetResult = reader.ReadToEnd()
    GetResult
+++++++++++++++++++++++++++++

I want to know which method is better to get the binary data directly from the response stream, not to save it into a file on my local hard disk and then get it out of file, then process.
Can I use the response stream directly? If I have to do some conversion, please show me the code.
Thank you very much again.
John

By on 11/23/2009 11:08 AM ()

The string returned by that method does not save it to your HD ...Its in memory and you can use it directly

Also html (WebRequest) does not support binary data .. period. So im assuming this is html text which looks like binary however this is a terribly inefficent format ..

eg

{00 00 00 00 01 0F 9F 75}

will take 25 to 50 bytes to store 8. Why not just use uuencode ?

By on 11/29/2009 4:36 PM ()

WebRequest is for arbitrary URI requests, such as HTTP or FTP. Although most commonly people are using the HttpWebRequest. An HTTP request can most certainly deliver any binary format desired. The StreamReader class reads text, so maybe that was what you are referring to.

By on 11/29/2009 5:12 PM ()

Yes i stand corrected there though it was HttpWebRequest.

Though i do believe his raw format is html or some sort of text representation as he has brackets and formatting

By on 11/29/2009 9:51 PM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper