Arc Forumnew | comments | leaders | submitlogin
1 point by akkartik 4553 days ago | link | parent

A 150k plaintext file works but a 147k binary pdf does not.

Update: The bug has to do with reading in bytes vs characters. Earlier, srv would readc from the POST body unless type was multipart, and your code (like the webupload example) would readb from the body otherwise. Now srv is always the one reading body, and it always reads characters using readc. As a result it gets confused by binary uploads that don't translate to characters.

Update 2: The sentence beginning 'Earlier' is incorrect, and webupload was always using readc as well. I'm not sure what I was looking at.



1 point by akkartik 4553 days ago | link

  $ ls -l bintest
  -rw-r--r-- 1 akkartik akkartik 145974 Jun  5 12:39 bintest
  $ racket -f as.scm
  arc> (w/infile f "bintest" (len:readbytes 200000 f))
  145974
  arc> (w/infile f "bintest" (len:readchars 200000 f))
  141878
readchars does some interpretation but otherwise works fine. However when trying to upload bintest through a socket, it never terminates. Most curious.

Update: I've confirmed that the bytes in the file fail to be encoded as a unicode string, so presumably that's the issue. Another bit of sloppiness is that we're trying to read n characters from the request body where n is the Content-Length in bytes. webupload.arc has always had this problem.

-----

2 points by lark 4553 days ago | link

This seems consistent with me having to use readb rather than readc to get webupload.arc to work before.

-----

1 point by akkartik 4553 days ago | link

Ok, try it out after a git pull.

http://github.com/nex3/arc/commit/b3d97ffc91

Your existing code won't work as is. Since it's meaningless to try to convert possibly-binary data to a string, file contents are now returned as a list of bytes. There's a helper called bytes-string for when you're sure you have just ascii data. Otherwise you'll need to know the encoding of text uploads and convert the bytes appropriately.

I should warn you that it's gotten a lot slower. You might need to temporarily up threadlife. I have some ideas for speeding it up, but let's check first if this works for you.

-----

2 points by lark 4549 days ago | link

lib/form-tests.arc seems to work. It does take over a minute to upload a 415kb file over localhost.

I see the following error:

  Can't coerce  (98 98 83 116 88 110 82 115 98 103 . nil) sym
Should there be a working example in form-tests.arc that uses bytes-string?

-----

1 point by akkartik 4549 days ago | link

Ok, do a git pull for an example.

http://github.com/nex3/arc/commit/46e3820a6b

Can you give more detail on how you ran into that error? I'm sure there are still bugs.

-----

3 points by lark 4549 days ago | link

Here's a testcase:

  (mac usform (n s f . body)
   (w/uniq ga
     `(tag (form method 'post action fnurl* class (string ,s) id ,n name ,n
                enctype "multipart/form-data")
       (fnid-field (fnid (fn (,ga)
                           (prn)
                           (,f ,ga))))
       ,@body)))

  (defop || req
    (usform "this-is-a-bug" "" [ (pr "args are " _) ]
          (tab (row "price: " (input "price"))
               (submit))))

  (def main ()
     (asv))

-----

1 point by akkartik 4549 days ago | link

Ah, this is because the fnid field is being read as a list of bytes. I could convert fnid to string as a special case. Another option is to pass it in with the action url like in aform-multi: http://github.com/nex3/arc/blob/46e3820a6b/lib/srv.arc#L560

Update: Ok, I finally found a way to check when it's safe to convert to string, so now all fields (including fnid) will auto-convert to string when possible. If you get a list of bytes you know it has some sort of non-ascii data.

http://github.com/nex3/arc/commit/d5331ac897

But it's probably made things a tad slower still (let me know). Sucks that reading characters in scheme just hangs when it should throw an error.

-----

1 point by akkartik 4543 days ago | link

Eek, there was still a bug in that. Here's the corrected commit: http://github.com/nex3/arc/commit/1882b3b589

I've also made it faster; 4x faster in my experiments. http://github.com/nex3/arc/commit/16173ec1ff

Do a git pull and give it a whirl.

-----