Back to Questions
928
views3
comments
Changes in FParsec?
f#

Hi everyone,

I was updating an email validation function I originally wrote on F# 1.9.6.x to F# 2.0 and FParsec trunk and noticed a few failing tests that weren't failing before this update. They all seem to be related to newlines, for example:

Failed: ID "152": "Test.
Folding.
Whitespace@example.com"
Expected: true
Actual: false
----
Failed: ID "159": ""test\
blah"@example.com"
Expected: false
Actual: true
----
Failed: ID "160": ""test
blah"@example.com"
Expected: true
Actual: false
----
Failed: ID "172": "first.(
middle
)last@example.com"
Expected: true
Actual: false
----
Failed: ID "174": "first(Welcome to
the ("wonderful" (!)) world
of email)@example.com"
Expected: true
Actual: false

The changes in my code are minimal, here's the diff.

Any ideas? Let me know if you need any other information.

Thanks
Mauricio

.

Hi Mauricio,

Could you please post a complete code sample reproducing the issue, including the test input and expected output, so that I can look into it tomorrow? Thanks.

By on 5/16/2010 1:55 PM ()

I just saw that you're using`pchar '\r'` and `pchar '\n'` to parse carriage return and line feed characters. A while ago, all FParsec parsers started to normalize any of the 3 standard newline sequences "\n", "\r\n" and "\r" to the single character '\n'. Hence both parsers `pchar '\r'` and `pchar '\n'` are now equivalent to the `newline` parser and will parse the "\r\n" as a single '\n'.

If you require the old behaviour, you could use the following raw char parser:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

let rawChar c : Parser<_,_> =
    match c with
    | '\n' ->
        let error = expectedStringError "\n"
        fun state ->
            if state.Iter.Read() = '\n' then
                Reply('\n', state.Advance(1, 1, 0)) // register newline
            else
                Reply(Error, error, state)
    | '\r' ->
        let error = expectedStringError "\r"
        fun state ->
            let cs = state.Iter.Read2()
            if cs.Char0 = '\r' then
                // only register a newline if '\r' is not followed by '\n'
                let newState = if cs.Char1 = '\n' then state.Next
                               else state.Advance(1, 1, 0)
                Reply('\r', newState)
            else
                Reply(Error, error, state)
    | _ -> 
         pchar c // just use the normal parser

This parser doesn't normalize newlines but should still register them properly, so that generated error messages come with a correct line and column count.

By on 5/16/2010 11:25 PM ()

Thanks Stephan! Yeah, I just noticed the all-caps announcement of this change in the changes.txt file, I should have RTFM ;-)

Cheers,

Mauricio

By on 5/17/2010 11:49 AM ()

    Topic tags

    IntelliFactory Offices Copyright (c) 2011-2012 IntelliFactory. All rights reserved.
    Home | Products | Consulting | Trainings | Blogs | Jobs | Contact Us | Terms of Use | Privacy Policy | Cookie Policy     		Built with WebSharper