[VB.NET/yEnc]

[CMG]

Helleu,

Ben bezig met pogin #3 om maar weer eens een NNTP client te schrijven. Het ging allemaal best aardig, yEnc encoder en decoder geschreven, met sockets NNTP verbinding opgezet en het encoden/decoden van yEnc files volgens de specificatie op yenc.org lukt prima (zowel single als multi-part messages).

Nu ben ik bezig om .nzb files te gebruiken om live multi-part yEnc files te decoden en samen te voegen op disk, maar op de een of andere manier heb ik last van 'spook' bytes in mijn decoder. Ik houd meer bytes over dan ik zou moeten, maar de files kloppen byte voor byte wel met op het eind NULL bytes. Dit gebeurd los per part, dus niet verdeeld over meerder messages. Het rare is dat dit enkel optreed bij messages die onderdeel vormen van een multi-part message.

Connected Event
Requesting Msg With Id nO4Am.50639$[e-mail]
Downloaded Article With ID <nO4Am.50639$[e-mail]>
CRC32: 942d216f
Complely downloaded file 00-editors-papillon-(ep)-2009.m3u
Writing Part 1 (CRC32 = 942d216f, Expected: 942d216f
Requesting Msg With Id nO4Am.29215$[e-mail]
Downloading Message With ID <nO4Am.29215$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <nO4Am.29215$[e-mail]>
CRC32: 5aee454d
Complely downloaded file 00-editors-papillon-(ep)-2009.nfo
Writing Part 1 (CRC32 = 5aee454d, Expected: 5aee454d
Requesting Msg With Id nO4Am.50640$[e-mail]
Downloading Message With ID <nO4Am.50640$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <nO4Am.50640$[e-mail]>
CRC32: 244c2810
Complely downloaded file 00-editors-papillon-(ep)-2009.sfv
Writing Part 1 (CRC32 = 244c2810, Expected: 244c2810
Requesting Msg With Id pO4Am.50641$[e-mail]
Downloading Message With ID <pO4Am.50641$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <pO4Am.50641$[e-mail]>
CRC32 Mismatch (Expected: fb569bad, Actual: f763b972)!
Requesting Msg With Id qO4Am.50642$[e-mail]
Downloading Message With ID <qO4Am.50642$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <qO4Am.50642$[e-mail]>
CRC32 Mismatch (Expected: da31553d, Actual: 4423e40e)!
Requesting Msg With Id sO4Am.50643$[e-mail]
Downloading Message With ID <sO4Am.50643$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <sO4Am.50643$[e-mail]>
CRC32 Mismatch (Expected: 2cbb682d, Actual: 9fad33fa)!
Requesting Msg With Id tO4Am.50644$[e-mail]
Downloading Message With ID <tO4Am.50644$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <tO4Am.50644$[e-mail]>
CRC32 Mismatch (Expected: 964425ae, Actual: 8278d3f6)!
Requesting Msg With Id uO4Am.50645$[e-mail]
Downloading Message With ID <uO4Am.50645$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <uO4Am.50645$[e-mail]>
CRC32 Mismatch (Expected: 60142d3f, Actual: fb7f8f43)!
Requesting Msg With Id wO4Am.50646$[e-mail]
Downloading Message With ID <wO4Am.50646$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <wO4Am.50646$[e-mail]>
CRC32 Mismatch (Expected: 97bb3840, Actual: e50917a6)!
Requesting Msg With Id xO4Am.50648$[e-mail]
Downloading Message With ID <xO4Am.50648$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <xO4Am.50648$[e-mail]>
CRC32 Mismatch (Expected: f857c3b5, Actual: 146ef0c2)!
Requesting Msg With Id zO4Am.50649$[e-mail]
Downloading Message With ID <zO4Am.50649$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <zO4Am.50649$[e-mail]>
CRC32 Mismatch (Expected: 290edcad, Actual: 033bc90e)!
Requesting Msg With Id BO4Am.50650$[e-mail]
Downloading Message With ID <BO4Am.50650$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <BO4Am.50650$[e-mail]>
CRC32 Mismatch (Expected: dc7740aa, Actual: b200f70d)!
Requesting Msg With Id DO4Am.50652$[e-mail]
Downloading Message With ID <DO4Am.50652$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <DO4Am.50652$[e-mail]>
CRC32 Mismatch (Expected: 84217ae6, Actual: 635004fc)!
Requesting Msg With Id DO4Am.50653$[e-mail]
Downloading Message With ID <DO4Am.50653$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <DO4Am.50653$[e-mail]>
CRC32 Mismatch (Expected: b7f60adb, Actual: 557c4aa1)!
Complely downloaded file 01-editors-papillon_(radio_mix).mp3
Writing Part 1 (CRC32 = f763b972, Expected: fb569bad
Writing Part 2 (CRC32 = 4423e40e, Expected: da31553d
Writing Part 3 (CRC32 = 9fad33fa, Expected: 2cbb682d
Writing Part 4 (CRC32 = 8278d3f6, Expected: 964425ae
Writing Part 5 (CRC32 = fb7f8f43, Expected: 60142d3f
Writing Part 6 (CRC32 = e50917a6, Expected: 97bb3840
Writing Part 7 (CRC32 = 146ef0c2, Expected: f857c3b5
Writing Part 8 (CRC32 = 033bc90e, Expected: 290edcad
Writing Part 9 (CRC32 = b200f70d, Expected: dc7740aa
Writing Part 10 (CRC32 = 635004fc, Expected: 84217ae6
Writing Part 11 (CRC32 = 557c4aa1, Expected: b7f60adb
Requesting Msg With Id pO4Am.29216$[e-mail]
Downloading Message With ID <pO4Am.29216$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <pO4Am.29216$[e-mail]>
CRC32 Mismatch (Expected: 59c0f700, Actual: ae41d553)!
Requesting Msg With Id rO4Am.29217$[e-mail]
Downloading Message With ID <rO4Am.29217$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <rO4Am.29217$[e-mail]>
CRC32 Mismatch (Expected: 4139e020, Actual: 12b37a59)!
Requesting Msg With Id uO4Am.29218$[e-mail]
Downloading Message With ID <uO4Am.29218$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <uO4Am.29218$[e-mail]>
CRC32 Mismatch (Expected: 6a03bfc7, Actual: 84589120)!
Requesting Msg With Id xO4Am.29219$[e-mail]
Downloading Message With ID <xO4Am.29219$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <xO4Am.29219$[e-mail]>
CRC32 Mismatch (Expected: 3a9327f7, Actual: 28b850ea)!
Requesting Msg With Id AO4Am.29221$[e-mail]
Downloading Message With ID <AO4Am.29221$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <AO4Am.29221$[e-mail]>
CRC32 Mismatch (Expected: 3f93ca6c, Actual: 89c6c524)!
Requesting Msg With Id DO4Am.29222$[e-mail]
Downloading Message With ID <DO4Am.29222$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <DO4Am.29222$[e-mail]>
CRC32 Mismatch (Expected: 0845aecb, Actual: aa3d0a74)!
Requesting Msg With Id GO4Am.29223$[e-mail]
Downloading Message With ID <GO4Am.29223$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <GO4Am.29223$[e-mail]>
CRC32 Mismatch (Expected: 35f308b4, Actual: f8cbb189)!
Requesting Msg With Id GO4Am.29224$[e-mail]
Downloading Message With ID <GO4Am.29224$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <GO4Am.29224$[e-mail]>
CRC32 Mismatch (Expected: e422da93, Actual: 713e3bbf)!
Complely downloaded file 02-editors-eat_raw_meat_blood_drool_(demo).mp3
Writing Part 1 (CRC32 = ae41d553, Expected: 59c0f700
Writing Part 2 (CRC32 = 12b37a59, Expected: 4139e020
Writing Part 3 (CRC32 = 84589120, Expected: 6a03bfc7
Writing Part 4 (CRC32 = 28b850ea, Expected: 3a9327f7
Writing Part 5 (CRC32 = 89c6c524, Expected: 3f93ca6c
Writing Part 6 (CRC32 = aa3d0a74, Expected: 0845aecb
Writing Part 7 (CRC32 = f8cbb189, Expected: 35f308b4
Writing Part 8 (CRC32 = 713e3bbf, Expected: e422da93
Requesting Msg With Id sO4Am.17067$[e-mail]
Downloading Message With ID <sO4Am.17067$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <sO4Am.17067$[e-mail]>
CRC32 Mismatch (Expected: 092e52ca, Actual: 6443270f)!
Requesting Msg With Id tO4Am.17068$[e-mail]
Downloading Message With ID <tO4Am.17068$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <tO4Am.17068$[e-mail]>
CRC32 Mismatch (Expected: 66cbb9a4, Actual: 47111b5b)!
Requesting Msg With Id vO4Am.17069$[e-mail]
Downloading Message With ID <vO4Am.17069$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <vO4Am.17069$[e-mail]>
CRC32 Mismatch (Expected: ef375b75, Actual: e5d4e28a)!
Requesting Msg With Id xO4Am.17070$[e-mail]
Downloading Message With ID <xO4Am.17070$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <xO4Am.17070$[e-mail]>
CRC32 Mismatch (Expected: 8048ac68, Actual: d80a661a)!
Requesting Msg With Id AO4Am.17071$[e-mail]
Downloading Message With ID <AO4Am.17071$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <AO4Am.17071$[e-mail]>
CRC32 Mismatch (Expected: f28e839f, Actual: b953f304)!
Requesting Msg With Id BO4Am.17073$[e-mail]
Downloading Message With ID <BO4Am.17073$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <BO4Am.17073$[e-mail]>
CRC32 Mismatch (Expected: e6c6f12e, Actual: 7fe7c892)!
Requesting Msg With Id CO4Am.17074$[e-mail]
Downloading Message With ID <CO4Am.17074$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <CO4Am.17074$[e-mail]>
CRC32 Mismatch (Expected: 7a265869, Actual: fe53a55d)!
Complely downloaded file 03-editors-like_treasure_(demo).mp3
Writing Part 1 (CRC32 = 6443270f, Expected: 092e52ca
Writing Part 2 (CRC32 = 47111b5b, Expected: 66cbb9a4
Writing Part 3 (CRC32 = e5d4e28a, Expected: ef375b75
Writing Part 4 (CRC32 = d80a661a, Expected: 8048ac68
Writing Part 5 (CRC32 = b953f304, Expected: f28e839f
Writing Part 6 (CRC32 = 7fe7c892, Expected: e6c6f12e
Writing Part 7 (CRC32 = fe53a55d, Expected: 7a265869
Requesting Msg With Id EO4Am.17076$[e-mail]
Downloading Message With ID <EO4Am.17076$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <EO4Am.17076$[e-mail]>
CRC32 Mismatch (Expected: 0bc0ee02, Actual: 5cc4631b)!
Requesting Msg With Id FO4Am.17077$[e-mail]
Downloading Message With ID <FO4Am.17077$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <FO4Am.17077$[e-mail]>
CRC32 Mismatch (Expected: 6e6f10cb, Actual: c285cc45)!
Requesting Msg With Id HO4Am.17078$[e-mail]
Downloading Message With ID <HO4Am.17078$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <HO4Am.17078$[e-mail]>
CRC32 Mismatch (Expected: a1c406ec, Actual: 769b99a7)!
Requesting Msg With Id JO4Am.17079$[e-mail]
Downloading Message With ID <JO4Am.17079$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <JO4Am.17079$[e-mail]>
CRC32 Mismatch (Expected: d56d32c2, Actual: 8798a590)!
Requesting Msg With Id LO4Am.17080$[e-mail]
Downloading Message With ID <LO4Am.17080$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <LO4Am.17080$[e-mail]>
CRC32 Mismatch (Expected: 87e815a3, Actual: 5be9d8eb)!
Requesting Msg With Id MO4Am.17081$[e-mail]
Downloading Message With ID <MO4Am.17081$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <MO4Am.17081$[e-mail]>
CRC32 Mismatch (Expected: fcc92ee1, Actual: ad0d0954)!
Requesting Msg With Id OO4Am.17082$[e-mail]
Downloading Message With ID <OO4Am.17082$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <OO4Am.17082$[e-mail]>
CRC32 Mismatch (Expected: ebe7ea30, Actual: 46545a53)!
Requesting Msg With Id PO4Am.17083$[e-mail]
Downloading Message With ID <PO4Am.17083$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <PO4Am.17083$[e-mail]>
CRC32 Mismatch (Expected: dec2ed82, Actual: 65d0ee58)!
Requesting Msg With Id RO4Am.17084$[e-mail]
Downloading Message With ID <RO4Am.17084$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <RO4Am.17084$[e-mail]>
CRC32 Mismatch (Expected: 63fd0b0b, Actual: c9d86256)!
Requesting Msg With Id TO4Am.17085$[e-mail]
Downloading Message With ID <TO4Am.17085$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <TO4Am.17085$[e-mail]>
CRC32 Mismatch (Expected: cfb9a80e, Actual: 27c9b794)!
Requesting Msg With Id TO4Am.17086$[e-mail]
Downloading Message With ID <TO4Am.17086$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <TO4Am.17086$[e-mail]>
CRC32 Mismatch (Expected: 983b275c, Actual: bcde7939)!
Complely downloaded file 04-editors-papillon_(tiesto_remix_radio_edit).mp3
Writing Part 1 (CRC32 = 5cc4631b, Expected: 0bc0ee02
Writing Part 2 (CRC32 = c285cc45, Expected: 6e6f10cb
Writing Part 3 (CRC32 = 769b99a7, Expected: a1c406ec
Writing Part 4 (CRC32 = 8798a590, Expected: d56d32c2
Writing Part 5 (CRC32 = 5be9d8eb, Expected: 87e815a3
Writing Part 6 (CRC32 = ad0d0954, Expected: fcc92ee1
Writing Part 7 (CRC32 = 46545a53, Expected: ebe7ea30
Writing Part 8 (CRC32 = 65d0ee58, Expected: dec2ed82
Writing Part 9 (CRC32 = c9d86256, Expected: 63fd0b0b
Writing Part 10 (CRC32 = 27c9b794, Expected: cfb9a80e
Writing Part 11 (CRC32 = bcde7939, Expected: 983b275c
Requesting Msg With Id DO4Am.50654$[e-mail]
Downloading Message With ID <DO4Am.50654$[e-mail]> from Group alt.binaries.sounds.mp3.complete_cd
Downloaded Article With ID <DO4Am.50654$[e-mail]>
CRC32: 2610c933
Complely downloaded file Editors-Papillon-(EP)-2009-SiRE.par2
Writing Part 1 (CRC32 = 2610c933, Expected: 2610c933

Als ik dan mijn controle functie run:

' Single Part Files
Byte Compare Summary for file 00-editors-papillon-(ep)-2009.m3u: 0/169 bytes are bad.
Byte Compare Summary for file 00-editors-papillon-(ep)-2009.nfo: 0/1173 bytes are bad.
Byte Compare Summary for file 00-editors-papillon-(ep)-2009.sfv: 0/209 bytes are bad.

' Multi-Part files
Lengths do not match for file 01-editors-papillon_(radio_mix).mp3: Original: 6483010, Our File: 6483278
Byte Compare Summary for file 01-editors-papillon_(radio_mix).mp3: 0/6483010 bytes are bad.
Our file has 268 bytes more then it should

Lengths do not match for file 02-editors-eat_raw_meat_blood_drool_(demo).mp3: Original: 4613664, Our File: 4613849
Byte Compare Summary for file 02-editors-eat_raw_meat_blood_drool_(demo).mp3: 0/4613664 bytes are bad.
Our file has 185 bytes more then it should

Lengths do not match for file 03-editors-like_treasure_(demo).mp3: Original: 4120983, Our File: 4121149
Byte Compare Summary for file 03-editors-like_treasure_(demo).mp3: 0/4120983 bytes are bad.
Our file has 166 bytes more then it should

Lengths do not match for file 04-editors-papillon_(tiesto_remix_radio_edit).mp3: Original: 6475277, Our File: 6475478
Byte Compare Summary for file 04-editors-papillon_(tiesto_remix_radio_edit).mp3: 0/6475277 bytes are bad.
Our file has 201 bytes more then it should

Controle functie code:

Visual Basic .NET:

    'backup is de map waar ik de downloads van mijn eigen prog heb staan.
        For Each FI As IO.FileInfo In New IO.DirectoryInfo("C:\yEnc\orig").GetFiles
            Test3(FI.FullName, IO.Path.Combine(FI.Directory.Parent.FullName & "\backup", FI.Name))
        Next

...
    Sub Test3(ByVal Original As String, ByVal NewFile As String)
        If Not IO.File.Exists(Original) Then
            System.Diagnostics.Debug.WriteLine(String.Format("Original File '{0}' does not exist", Original))
            Return
        End If
        If Not IO.File.Exists(NewFile) Then
            System.Diagnostics.Debug.WriteLine(String.Format("New File '{0}' does not exist", NewFile))
            Return
        End If
        Dim FileName As String = New IO.FileInfo(Original).Name
        Dim oDecoded, nDecoded As Byte()
        Using BW As New IO.BinaryWriter(New IO.FileStream(Original, IO.FileMode.Open))
            ReDim oDecoded(BW.BaseStream.Length - 1)
            BW.Write(oDecoded, 0, oDecoded.Length)
            BW.Flush()
            BW.Close()
        End Using
        Using BW As New IO.BinaryWriter(New IO.FileStream(NewFile, IO.FileMode.Open))
            ReDim nDecoded(BW.BaseStream.Length - 1)
            BW.Write(nDecoded, 0, nDecoded.Length)
            BW.Flush()
            BW.Close()
        End Using
        If oDecoded.Length <> nDecoded.Length Then
            System.Diagnostics.Debug.WriteLine(String.Format("Lengths do not match for file {2}: Original: {0}, Our File: {1}", oDecoded.Length, nDecoded.Length, FileName))
        End If
        Dim Min As Long = oDecoded.Length
        If nDecoded.Length < Min Then Min = nDecoded.Length
        Dim MisMatchCount As Long = 0
        For i As Long = 0 To Min - 1
            If oDecoded(i) <> nDecoded(i) Then
                MisMatchCount += 1
                System.Diagnostics.Debug.WriteLine(String.Format("Byte {0} does not match up ({5}/{6} bytes are bad (checked {7}/{6})). Original: '{1}' ({2}), New: '{3}' ({4})  ", i, ChrW(oDecoded(i)), oDecoded(i), ChrW(nDecoded(i)), nDecoded(i), MisMatchCount, Min, i))
            End If
        Next
        System.Diagnostics.Debug.WriteLine(String.Format("Byte Compare Summary for file {0}: {1}/{2} bytes are bad.", FileName, MisMatchCount, Min))
        For i As Long = Min To oDecoded.Length - 1
            System.Diagnostics.Debug.WriteLine(String.Format("Byte {0} is missing from the new file: '{1}' ({2})  ", i, ChrW(oDecoded(i)), oDecoded(i)))
        Next
        If nDecoded.Length > Min Then
            System.Diagnostics.Debug.WriteLine(String.Format("Our file has {0} bytes more then it should", nDecoded.Length - Min))
        End If
        'For i As Long = Min To nDecoded.Length - 1
        '    System.Diagnostics.Debug.WriteLine(String.Format("Byte {0} is missing from the original file: '{1}' ({2})  ", i, ChrW(nDecoded(i)), nDecoded(i)))
        'Next
    End Sub

Het rare is dat als ik de multi-part test doe met de files van yEnc.org dat het wel werkt.

Iemand enig idee waar dit in kan zitten of dat het misschien een of andere niet-standaard implementatie is?

Groet,

Nick.

[CMG]

Ok, heb het zelf opgelost.

Het probleem was dat de msg door een slechte yEnc Encoder ge-encode is. Berichtregels die beginnen met een punt (chr(46)) worden altijd gevolgd door nog een punt en dient dan als een char gezien te worden. Een goeie encoder had de punt op de begin regel ge-escaped...

Ow ja, de test functie hier is ook niet in orde aangezien hij de files overschrijft ipv leest Bad Copy/Paste