mostlylucid

scott galloway's personal blog...
posts - 911, comments - 723, trackbacks - 11

My Links

News

Archives

Post Categories

Misc. Coding

ViewState compression using BZip2...latest prototype...

Just a first stab at this, uses some classes I used in another project (I'll attribute the compression code later...I believe I got it from the Sharpziplib stuff - with some modifications - I'll stick the comment headers back in to comply with the Licensing stuff later - right now consider it an example only) the actual code to compress the viewstate is VERY simple:

using System;
using System.Web.UI;
using System.IO;
namespace ViewStateCompression
{
    /// 
    /// Summary description for CompressedVSBasePage.
    /// 
    public class CompressedVSBasePage :System.Web.UI.Page
    {
        private LosFormatter  _formatter = new LosFormatter();
        protected override void SavePageStateToPersistenceMedium(object viewState)
        {
            StringWriter sw = new StringWriter();
            _formatter.Serialize(sw, viewState);
            string outStr = Compression.Compress(sw.ToString());
            Page.RegisterHiddenField("__COMPRESSEDVIEWSTATE",outStr);
        }
        protected override object LoadPageStateFromPersistenceMedium()
        {
            string vsString = Request.Form["__COMPRESSEDVIEWSTATE"];
            string outStr = Compresssion.DeCompress(vsString);
            return _formatter.Deserialize(outStr);
        }
    }
}

To use it, just inherit from this page instead of the normal System.Web.UI.Page. Obviously you lose some of the normal Viewstate functions such as encryption - but these should be easy to slot back in...I am currently seeing pretty large savings in ViewState size from using this - using BZip2 compression - as it's a piece of code I had lying about..., please try it out. Any comments / suggestions are, as always, appreciated.

UPDATE (27/05/2004):  I've updated the demo project, you can now download this from here. Main changes are that it now uses my Compression helper object - this uses the standard SharpZipLib, is a bit more efficient and lets you switch between all the SharpZipLib compression types very easily. My experience has shown that this method is best when you're using objects like DataSets which under.1.1 serialize pretty poorly (as in DataSets really serialize to a form of XML - which tends to compress really well). In addition, do not turn on encryption when doing this - the compression is REALLY poor then. It's really simple to add encryption back in again - just do it to the string you get from the Stringwriter after serialization - I'll add an example later (using Rijndael is also gonna be more secure and much faster than 3DES).  Also, someone called 'Mark' left a comment with some great performance figures, here's the comment:

Played around with this a little. For smaller view states (did a short comparison with a view state of 100200 bytes) it's faster on a normal network not to use GZip or BZip2. Roughly: The original file size was 224 Kb. GZip got the file size down to 197 Kb but it took about 0.5 seconds longer, BZip2 got the file size down to 190 Kb, but it took about 1.3 seconds longer. With a larger view state (used a view state of 1115624 bytes) the original file size was 893 Kb. GZip got the file size down to 501 Kb and was actually almost 0.4 seconds faster. The BZip2 got the file size down to 434 Kb, but it was still about 1 second slower than using no compression.

These figures really don't mean much, since I didn't take many samples. However, for me it confirmed that BZip2 might only really make it go faster when the view state is very very big. BZip2 provides very nice compression, but it's slower. GZip doesn't provide as high as a compression rate as BZip2, but it's faster. Below a certain threshold (which is hard to determine since network speed influences this), compression really just slows it down. I am looking at using a fast compression algorithm like LZO. Hope this helps anyone interested in this.

Print | posted on Saturday, January 03, 2004 7:26 PM | Filed Under [ .NET ASP.NET Code Snippets ]

Feedback

Gravatar

# re: ViewState compression using BZip2...latest prototype...

Just trying out your example. One problem though, what is ComDePress?
1/15/2004 3:50 PM | Ian
Gravatar

# re: ViewState compression using BZip2...latest prototype...

Scratch the last question. Didn't notice your test project.
1/15/2004 3:58 PM | Ian
Gravatar

# re: ViewState compression using BZip2...latest prototype...

Played around with this a little. For smaller view states (did a short comparison with a view state of 100200 bytes) it's faster on a normal network not to use GZip or BZip2. Roughly: The original file size was 224 Kb. GZip got the file size down to 197 Kb but it took about 0.5 seconds longer, BZip2 got the file size down to 190 Kb, but it took about 1.3 seconds longer. With a larger view state (used a view state of 1115624 bytes) the original file size was 893 Kb. GZip got the file size down to 501 Kb and was actually almost 0.4 seconds faster. The BZip2 got the file size down to 434 Kb, but it was still about 1 second slower than using no compression.

These figures really don't mean much, since I didn't take many samples. However, for me it confirmed that BZip2 might only really make it go faster when the view state is very very big. BZip2 provides very nice compression, but it's slower. GZip doesn't provide as high as a compression rate as BZip2, but it's faster. Below a certain threshold (which is hard to determine since network speed influences this), compression really just slows it down. I am looking at using a fast compression algorithm like LZO. Hope this helps anyone interested in this.
2/23/2004 6:03 PM | Mark
Gravatar

# re: ViewState compression using BZip2...latest prototype...

Thanks for that - very interesting stuff...also part of the reason I abandoned this approach - IIS 6.0 compression / Ben Lowery's compression module is much easier (and you can get savings on the whole page, not just ViewState).
I am currently playing around with another, considerablymore advanced way of doing this - using a proxy handler factory in front of the original ASP.NET one. Problem is that compressing the eventual output of the LOSFormatter is only half the battle - if you encrypt it obviously compression does very little (and occasionally will actually increase the size). Only way seems to be to provide a completely different implementation of ViewState - whuch is what I'm currently attempting (using an Abstract Factory to allow you to determine where you want ViewState stored, what compression / encryption / hashing algorithm you want applied and then plug in your own whizzy store at will). This is one of these 'probably never finish' type projects, but it sure would be nice if I can get it working appropriately (by which time I dare say I'll have to change it to account for control state). This is also based on some work in this here blog engine - using proxy handlers to add functionality to base ASP.NET - which I suggest everyone check out!
2/23/2004 9:14 PM | Scott Galloway
Gravatar

# re: ViewState compression using BZip2...latest prototype...

I have noticed Ben Lowery's compression module. Doesn't this require a client that supports compression?
2/23/2004 10:50 PM | Mark
Gravatar

# re: ViewState compression using BZip2...latest prototype...

Yup, but it does auto-detect, clients send out a special header which inform the web server if they support compression and what types they support, the module is then able to detect this and apply compression only is that is true.
All current browsers support compression (I believe it is part of the HTTP 1.1 specification - to support that you must support compression)
2/24/2004 10:36 AM | Scott Galloway
Gravatar

# Better than whole http compression

The reason this is much better is that most of the IE clients out there have massive bugs with opening compressed pages... but viewstate is just sent back to the server, so why not compress it? You get to save the bandwidth twice, during download AND once while the user's browser is uploading the request. All with no compatiblity problems? Sounds too good to be true
5/5/2004 1:35 PM | automatt
Gravatar

# More asp.net filters...

Where use questionable methods to compress/store/whatever the viewstate.
6/18/2004 3:46 PM | Philip Rieck
Gravatar

# .net compressed viewstate

At work I've been battling with large viewstates in .net. Mostly my viewstates will get large because I'm storing a datatable in it so it can be sorted without re-querying the sql database. A kind soul put together some C...
10/20/2004 6:22 PM | modernhippy
Gravatar

# ViewState

Ping Back??:blog.csdn.net
10/29/2004 1:40 AM | DuSDong
Gravatar

# re: ViewState compression using BZip2...latest prototype...

Thanks for the head start. You'll be happy to know that I figured out a very easy way to improve your algorithm for both speed and compression ratio.

Your big mistake was to compress the base64encoded data. If you unbase64encode it first then compress, then base64encode that, you'll get compression ratios of about 13-15% (at least on my test pages that I'm working on). On top of that, if you use GZIP and not BZIP2 the time it takes to compress is significantly faster.

On my crappy Atlhon 850 is was originally getting times of 300-500 ms to compress using BZIP2. When I switched to GZIP it went to under 10-60ms. Also, using this scheme the compression ratio between GZIP and BZIP2 seem to be close to the same.

To do this I just modified your Compress and DeCompress methods in your helper class. Here's the code.

public static string Compress(string stringToCompress)
{

//Assume string is already base64encoded;
byte[] buf = Convert.FromBase64String(stringToCompress);

byte[] compressedData = Compress(buf);

string strOut = Convert.ToBase64String(compressedData);

return strOut;
}


public static string DeCompress(string stringToDecompress)
{
string outString = string.Empty;

if (stringToDecompress == null)
{
throw new ArgumentNullException("stringToDecompress","You tried to use an empty string");
}

try
{

byte[] inArr = Convert.FromBase64String(stringToDecompress.Trim());

byte[] buf = DeCompress(inArr);

outString = Convert.ToBase64String(buf);

}
catch (NullReferenceException nEx)
{
return nEx.Message;
}

return outString;

}

Hope this helps.
1/5/2005 4:45 PM | Mark DeMichele
Gravatar

# re: ViewState compression using BZip2...latest prototype...

Nice one, I'll take a closer look at this when I return from holiday - and may include it in another post (been meaning to update this thing with an improved version I have lurking about).
1/5/2005 10:14 PM | Scott Galloway
Gravatar

# Why I'm not posting about ASP.NET

Why I'm not posting about ASP.NET
10/1/2007 12:00 AM | mostlylucid
Comments have been closed on this topic.

Powered by: