Back to Questions
1.4k
views3
comments
Async Workflow and Filestream
f#async

Hi,

With the use of async workflows, I was wondering what the best (or most common) way to handle mutliple writes to the disk was.

Should you have one thread for all file writes (which are done one after another) ? Should you prefer one thread per file write except that file writing would require access to a Semaphore, and if so, how many writes can be done without over-slowing the system ?

Also, if at the same time you perform file reads, should you assume that since it's just copy from a stream it is a separate issue, and should thus be used as 

1

let! x = read_in "file.txt" 0 150

, leaving the file writes aside ; or should the reads be done in a separate thread ? Or in several separate threads with a Semaphore (to avoid opening a lot of files at once) ? Should there be a Semaphore shared by writes and reads so that, basically you can have at most n writes, m reads, and x file accesses at once ?

Thanks a lot for your help !

Julien

.

Hi Julien,

My guess is that if the reads/writes are independent you should probably just create all the tasks and let the OS sort things out. How many tasks are you talking about here, and what kind of overall result (including performance results) are you trying to achieve?

Don

By on 11/14/2007 6:23 AM ()

Basically I am (lazily, painfully etc.) trying to make a BitTorrent client using F# concepts.

You are connected to several clients, but the number of the latter remains limited due to the tit-for-tat download/upload strategy that makes slow uploaders download slower (having multiple peers -> division of yyour upload speed -> download slower). From the different clients I've seen, it seems that you are generally connected to at most 10 users (5 to download from and upload to + 5 to upload to when you have the whole file)

I was assuming (perhaps wrongly) that even a low number of simultaneous I/O operations could significantly slow the machine.

I was hesitating between doing the I/O operations on a separate thread or on several ones.

In the former case, which I was about to go for, I was thinking about using a Mailbox to handle all the writes, and leave the reads be done normally (using the let! data = read_async "file" 0 100), but I wasn't sure if there was a better way to do that.

julien

By on 11/14/2007 6:44 AM ()

Interesting project!

I suspect that this will be an entirely I/O bound application and that you could start many network connections simultaneously and be only bound by the bandwidth you get (and how many connections you make before your ISP throws you off the network :-) ).

So at a first guess I think you can just do the writes simultaneously. Having multiple writes outstanding should not be a problem at all and should not cause any significant performance loss (apart from putting more burden on the network).

Please post what you find!

Kind regards

don

By on 11/14/2007 5:25 PM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper