[cgiapp] Data validation of file uploads
nicholas at periapt.co.uk
Mon Sep 20 10:08:20 EDT 2010
Comments below in no particular order
* Once processing is finished the data is presumably written to a file
or a database. and the memory can be reclaimed.
* One of my concerns about Data::FormValidator::Constraints::Upload and
Data::FormValidator::Filters::Image is that as far as I can see they
must be loading the entire file into memory at least temporarily. And
yet the user still has no access to the data or has not actually even
got a non -temporary file name.
* These sort of issues are the same that any successful website must
address. There are books on scaling websites (such as splitting your
server into a lightweight front-end caching proxying websever and a
modperl heavy back-end server.)
* One can use rate limiting, authentication and other techniques to
deter DOS attacks. In fact in my use cases upload will only be possible
for the beneficial owner of the website anyway.
* I've actually got my code working (though I am just about to try it
out in anger) so you can see more concretely what I am thinking of at
notice that this module uses CGI::Upload for the heavy lifting,
Data::FormValidator for the data validation, and
CGI::Application::Plugin::JSON for communicating back to the client.
Michael Peters wrote:
> On 09/20/2010 03:01 AM, Nicholas Bamber wrote:
>> For the purposes of my discussion we can assume that, say 5M, is a
>> reasonable maximum file size. The modules that I am comparing with do
>> slurp the entire file into memory. And by only providing a file handle
>> they force the caller to do it more than once.
> But even 5M is still a lot to have your processes grow by on each
> request. For instance, a common mod_perl setup with say 100 apache
> children could grow by half a gig really fast and result in total
> machine lockup. It's not an edge case to not want to expose yourself
> to a DOS attack.
More information about the cgiapp