Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Compressing data structures for cookies

by gwg (Beadle)
on May 08, 2007 at 21:49 UTC ( #614257=perlquestion: print w/replies, xml ) Need Help??

gwg has asked for the wisdom of the Perl Monks concerning the following question:

Greetings monks,

I would like to store some user settings in a cookie, and was wondering if anyone had any advice on methods for compressing and/or encoding the data, which would be in the form of a hash of arrays. Storing all possible settings using FreezeThaw to stringify the data gives a file of 402 bytes in size. The number and variety of compression and encoding modules is a bit overwhelming, so if anyone has any suggestions for a particularly suitable module, I would be most grateful!

Update / clarification:

I'm already using a local variant of CGI::Session to store data server-side. The problem is that there are two machines which run the program, and a user could be directed to either of the two (the sys admin says that each user is usually sent to the machine they were previously on, but it can't be guaranteed every time). The other thing is that the volume of users is such that it is not feasible to keep the session data for every person, and it would be better for users to be able to have their settings persist between sessions.

The data that I want to store isn't anything critical - there are five filters that users can set up for searching / viewing data, and it's these settings that I would like to put in the cookie. The cookie data would be compared against the list of valid parameters, so if users did have the inclination to tinker with the settings, the worst they could do would be to set the maximum possible filters, which you can do validly by setting those filters using the cgi.

Thanks,
gwg

  • Comment on Compressing data structures for cookies

Replies are listed 'Best First'.
Re: Compressing data structures for cookies
by Joost (Canon) on May 08, 2007 at 21:55 UTC
Re: Compressing data structures for cookies
by GrandFather (Saint) on May 08, 2007 at 22:19 UTC

    Store the data server side and use the cookie as a key to find it. CGI::Session pretty much wraps it all up for you.


    DWIM is Perl's answer to Gödel
Re: Compressing data structures for cookies
by Herkum (Parson) on May 08, 2007 at 23:15 UTC

    There are too many limitations when you use cookies to store data. There are size limits, it reduces the responsiveness of the client and it is just not secure.

    That is why people are going to tell you over and over that you need to use CGI::Session. Once you learn it you will be eternally grateful for it.

      I'm already using a local variant of CGI::Session to store data server-side. The problem is that there are two machines which run the program, and a user could be directed to either of the two (the sys admin says that each user is usually sent to the machine they were previously on, but it can't be guaranteed every time). The other thing is that the volume of users is such that it is not feasible to keep the session data for every person, and it would be better for users to be able to have their settings persist between sessions.

      I just re-checked the maximum size of the data structure as the original estimate had extra unneeded info in it; it's 402 bytes (as a file), and most user settings would probably be half that.

        If you are using CGI::Session with a database, having multiple machines would not be a problem. The reason for this is that the user would just a Session ID cookie. The session ID would correspond with a primary key on a 'sessions' table in the database. The database would contain all user settings and can store whatever information that they need for the duration of the session.

        All you would have to do is point both web services to the same database where the session data is stored. Now it would not matter which server you go to and you can add a third or a fourth server, etc... and you would not have to change anything.

        This would also apply to data as well. You can expand the number of user settings from your 402 bytes to 3 or 4 KB and you not have to change anything.

        The other thing is that the volume of users is such that it is not feasible to keep the session data for every person

        You have to keep the data somewhere. You stated that currently that the settings take up 402 bytes. Than means 1 MB can store settings for 2600 users. That is not much storage space at all.

        , and it would be better for users to be able to have their settings persist between sessions.

        If the data is persistent and is not needed for a particular session then store it in a table. You need someway of identifying the user, create a 'user' table and put everything in there.

        Databases are not a limitation, they can manage thousands and even millions of records and gigs of data.

        Cookies are what are limiting, move away from using them so much.

                                /- prog 1  -\
        client - load balancer -            - database
                                \- prog 2  -/
        
        
Re: Compressing data structures for cookies
by Bro. Doug (Monk) on May 08, 2007 at 22:41 UTC
    Try using Compress::Zlib . Once compressed, you'll probably want to base64 encode them. This will increase the size of your zip by 4/3, but will allow you to store 'binary' data on the cookie.
    use Compress::Zlib ; use Storable ; my %hash = &some_data ; my $data = pack( "u*", Compress::Zlib::MemGzip( freeze \%hash) ) ; # now you can do as you please with $data # to unthaw: my $hashref = thaw( Compress::Zlib::MemGunzip( unpack( "u", $data ) ) ) ;

    You'll also want to carefully manage the variables you need stored, as cookies tend to allow a minimal amount of space.

    Update: GrandFather is absolutely right, though. The best way to store the data is on the server. This method will work, if you're insistent.

    Another Update:
    I've been thinking about this, and I strongly advise you not to do it.

    If you give a variable to a user, that user may choose to unpack, unzip, and thaw it. Then that user could look at your data structure and do something to the cookie ( for example, set $data{'user'} = 'admin' ) and send it back to you. This could be disastrous.

    Peace monks,
    Bro. Doug :wq

      Thanks for the advice so far. The data that I want to store isn't anything critical - there are five filters that users can set up for searching / viewing data, and it's these settings that I would like to put in the cookie. The cookie data would be compared against the list of valid parameters, so if users did have the inclination to tinker with the settings, the worst they could do would be to set the maximum possible filters, which you can do validly by setting those filters using the cgi.

      One possibility if the data structure is too big might be to have a couple of cookies, as there are some filters which only apply to certain aspects of the program. I don't really want to be setting huge numbers of cookies if I can help it, though.

Re: Compressing data structures for cookies
by grinder (Bishop) on May 09, 2007 at 08:53 UTC
    I would like to store some user settings in a cookie

    Don't do that.

    Sooner or later, someone will figure out how to decode the cookie contents and give themselves an enhanced status that you were probably not willing to concede.

    Use the cookie as, well, a cookie. That is, a random string of bits that has no intrinsic meaning. When the client returns to your site, you read the cookie and go look up the settings that are keyed to that user in some table. Think of it as a the 100 dollar note torn in half and shared between two spies. Alone, either half is worthless: it's only when you put them together that they mean anything.

    A cardinal rule of web programming is to never trust the user. Don't put any semantic meaning into a cookie. Keep all that information on your server. You'll also keep you're cookie size down. 256 bits encoded in Base-64 takes 11 bytes, and I doubt you'll hit a collision before the heat death of the universe.

    • another intruder with the mooring in the heart of the Perl

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://614257]
Approved by Joost
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2021-06-21 06:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)












    Results (98 votes). Check out past polls.

    Notices?