Pivoting POCO generic lists using reflection

by martin 7. November 2011 22:05

I recently had to pivot data that I had in a list of POCO objects. I had to do this for a couple of different object types and it just felt wrong to have to recode something so generic over and over again. So... I figured I'd write a little helper class that utilizes reflection and allows me to pivot on any list of POCO objects, and specify what I want to pivot on (X and Y columns) and what I should be summing. This is all done using System.Collections.Generic and System.Reflection so it may not be the ideal solution from a performance perspective for huge datasets. On the other hand, for most datasets and modern hardware it isn't a problem at all, and it is really flexible and very highly reuseable for cranking out reports quickly.

Code Snippet

public class PivotResult
{
 
    public String[] ColumnNames;
    public String[] RowNames;
    public float[,] ValuesTable;
    public int Count;
}
public class ReflectionPivot
{
    public PivotResult Pivot(List<Object> ObjectList, String ColumnProperty, String RowProperty, String ValueProperty)
    {
        if (ObjectList == null)
            return null;
        if (ObjectList.Count == 0)
            return null;
        Object first = ObjectList[0];
        Type type = first.GetType();
        PropertyInfo[] properties = type.GetProperties();
        PropertyInfo colPropInfo = null;
        PropertyInfo rowPropInfo = null;
        PropertyInfo valPropInfo = null;
        foreach (PropertyInfo pInfo in properties)
        {
            if (pInfo.Name.Equals(ColumnProperty))
                colPropInfo = pInfo;
            if (pInfo.Name.Equals(RowProperty))
                rowPropInfo = pInfo;
            if (pInfo.Name.Equals(ValueProperty))
                valPropInfo = pInfo;
        }
        if (colPropInfo == null)
        {
            throw new ArgumentException("Column Property invalid.");
        }
        if (rowPropInfo == null)
        {
            throw new ArgumentException("Row Property invalid.");
        }
        if (valPropInfo == null)
        {
            if (!(ValueProperty.Equals("1")))
            {
                throw new ArgumentException("Value Property invalid.");
            }
        }
        List<String> lstColNames = new List<string>();
        List<String> lstRowNames = new List<string>();
        foreach (Object obj in ObjectList)
        {
            string strCol = colPropInfo.GetValue(obj, null).ToString();
            string strRow = rowPropInfo.GetValue(obj, null).ToString();
            if (!(lstColNames.Contains(strCol)))
            {
                lstColNames.Add(strCol);
            }
            if (!(lstRowNames.Contains(strRow)))
            {
                lstRowNames.Add(strRow);
            }
        }
        PivotResult pr = new PivotResult();
        pr.Count = 0;
        pr.ValuesTable = new float[lstRowNames.Count, lstColNames.Count];
        pr.RowNames = lstRowNames.ToArray();
        pr.ColumnNames = lstColNames.ToArray();
        foreach (Object obj in ObjectList)
        {
            float fVal = 0;
            if (ValueProperty.Equals("1"))
            {
                fVal = 1;
            }
            else
            {
                fVal = (float)valPropInfo.GetValue(obj, null);
            }
            pr.Count++;
            string strCol = colPropInfo.GetValue(obj, null).ToString();
            string strRow = rowPropInfo.GetValue(obj, null).ToString();
            int intColName = lstColNames.FindIndex(x => x == strCol);
            int intRowName = lstRowNames.FindIndex(x => x == strRow);
            pr.ValuesTable[intRowName, intColName] += fVal;
        }
        return pr;
    }
}

And you would use it something like this:

ReflectionPivot reflectionPivot = new ReflectionPivot();

PivotResult pivotResult = reflectionPivot.Pivot(intermediate.ToList<Object>(), "Team", "Grade", "1");

PivotResult pivotResult2 = reflectionPivot.Pivot(lstSales, "Division", "State", "Sales");

And then you can simply pass the PivotResult instance to a MVC view and use it to display a graph or to just show in a table list like this:

Code Snippet

<%@ Page Title="Sales Report" Language="C#" MasterPageFile="~/Views/Shared/Site.Master" Inherits="System.Web.Mvc.ViewPage<PivotResult>" %>


	
<asp:Content ID="Content2" ContentPlaceHolderID="MainContent" runat="server">
 
    <script src="<%: Url.Content("~/Scripts/visualize.jQuery.js") %>" type="text/javascript"></script>
    <script type="text/javascript">
//<![CDATA[
            $(function () {
                $(".jqvisualizesimple").visualize();
            });
        });
//]]>
    </script>
 
<h2>Sales Report</h2>
 
<table class="jqvisualizesimple">
<thead>
<td>&nbsp;</td>
<% foreach(var strCol in Model.ColumnNames) { %>
<th scope="col"><%: strCol %></th>
<% } %>
</thead>
<tbody>
    <% for (int i = 0; i < Model.ValuesTable.GetLength(0); i++)
       {
           var strRow =  Model.RowNames[i]; %>
           <tr><th scope="row"><%: strRow %></th>
       <%
           for (int j = 0; j < Model.ValuesTable.GetLength(1); j++)
           { %>
       <td><%:Model.ValuesTable[i, j]%></td>
    <%     }
       }%>
</tr>
</tbody>
</table>
</asp:Content>

I was really hoping to do this using pure LINQ, but it turned out to be the much more difficult route so I ended up backing off and using reflection instead. If someone has seen something like this done in LINQ, please do send me a message or a comment and point towards that solution as I would very much enjoy seeing it and learning from it.

Currently rated 3.0 by 5 people

Currently 3/5 Stars.
1
2
3
4
5

Tags: .net, c#

Writing Extended ASCII codes with BinaryWriter in .Net

by martin 12. October 2011 16:10

Ok, so this seems simple enough, but it took me a bit of Googling to figure out that I needed to pass the magic number 1252 to the GetEncoding method to get this to work correctly. ASCIIEncoding only encodes values from 0 to 127 correctly, but values 128 to 255 do not get written out properly. 1252 refers to the Window Code Page for the Latin Alphabet for Western languages, and is sometimes incorrectly referred to as ANSI. Hopefully this will save someone else some time searching for the solution.....

Code Snippet

String strEscapeCode = new String('\xC4', 5);
 
using (FileStream stream = new FileStream("TestFile.bin", FileMode.Create))
{              
    using (BinaryWriter writer = new BinaryWriter(stream, Encoding.GetEncoding(1252)))
    {
        writer.Write(strEscapeCode.ToCharArray());
        writer.Close();
    }
}

Currently rated 3.0 by 5 people

Currently 3/5 Stars.
1
2
3
4
5

Tags: c#, .net

Using Secure Hashing safely in .Net

by martin 5. October 2011 22:41

SHA 1 and SHA 256 are both great hashing algorithms, but both are susceptible to a Rainbow Table attack and with Moore's law and the great race by Google, Bing, and other search engines to index absolutely every bit of data out there on the internet, the chances of these attack succeeding grows every day. As a developer fending off this kind of vector, it is considered a best practice to not only always salt your data before hashing it, but to always make sure that the salt is completely random in nature. Additionally, since both SHA 1 and SHA 256 are Big - O(n) algorithms as far as performance cost is concerned and relatively inexpensive to execute on modern hardware, it is also a good idea to have multiple iterative calls to the hashing algorithm to come up with the hash. And while we are at it, we might as well switch up between using both algorithms just in case one has a yet undiscovered flaw similar to that which was discovered with MD-5 or even SHA-0 before that. Below is a C# implementation of a secure hash helper class to help you do all this. It provides a way to create a hash from a value and to confirm that a hash was made from a given value. It currently does additional Base64 encoding/decoding of the hash value for ease of use in web applications (tamper proofing forms against request forgery, storing password hashes, etc), but this can easily be removed if you'd prefer to deal in raw byte arrays.

Code Snippet

using System;
using System.Security.Cryptography;
 
namespace SecureHashTestConsole
{
    public static class SecureHashHelper
    {
        // set this to a relatively small number that you like.  (4 to 25 suggested range)  Larger number = longer random hash
        private const int defaultNumBytesForSeed = 9;
 
        // set this to a decent number, not too low, not to high (5 to 1000 suggested range)  Larger number = slower performance but less likely rainbow table cracking possibility
        private const int defaultIterations = 88;     
 
        public static bool CompareSecureHash(string TestHash, string TestValue, int numRandBytesForSeed = defaultNumBytesForSeed, int iterations = defaultIterations)
        {
            byte[] hashArray = Convert.FromBase64String(TestHash);
            byte[] derivedSalt = new byte[numRandBytesForSeed];
            Buffer.BlockCopy(hashArray, 0, derivedSalt, 0, numRandBytesForSeed);
            string strComp = CreateSecureHash(TestValue, derivedSalt, iterations);
            return strComp.Equals(TestHash);
        }
 
        public static string CreateSecureHash(byte[] sbuffer, int numRandBytesForSeed = defaultNumBytesForSeed, int iterations = defaultIterations)
        {
            RNGCryptoServiceProvider rand = new RNGCryptoServiceProvider();
            byte[] randSalt = new byte[numRandBytesForSeed];
            rand.GetBytes(randSalt);
            return CreateSecureHash(randSalt, sbuffer, iterations);
        }
 
        public static string CreateSecureHash(string ValueToHash, int numRandBytesForSeed = defaultNumBytesForSeed, int iterations = defaultIterations)
        {
            RNGCryptoServiceProvider rand = new RNGCryptoServiceProvider();
            byte[] randSalt = new byte[numRandBytesForSeed];
            rand.GetBytes(randSalt);
            System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
            Byte[] bytes = encoding.GetBytes(ValueToHash);
            return CreateSecureHash(randSalt, bytes, iterations);
        }
 
        private static string CreateSecureHash(string ValueToHash, byte[] randSalt, int iterations = defaultIterations)
        {
            System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
            Byte[] bytes = encoding.GetBytes(ValueToHash);
            return CreateSecureHash(randSalt, bytes, iterations);
        }
        private static string CreateSecureHash(byte[] randSalt, byte[] sbuffer, int iterations = defaultIterations)
        {
            SHA1Managed sha1 = new SHA1Managed();
            SHA256Managed sha256 = new SHA256Managed();
 
            byte[] buffer = new byte[randSalt.Length + sbuffer.Length];
            Buffer.BlockCopy(randSalt, 0, buffer, 0, randSalt.Length);
            Buffer.BlockCopy(sbuffer, 0, buffer, randSalt.Length, sbuffer.Length);
 
            for (int i = 0; i < iterations; i++)
            {
                // flip the order of using these two around if you want longer hashes 
                sha256.ComputeHash(buffer);
                buffer = sha256.Hash;
                sha1.ComputeHash(buffer);
                buffer = sha1.Hash;
            }
 
            byte[] output = new byte[buffer.Length + randSalt.Length];
            Buffer.BlockCopy(randSalt, 0, output, 0, randSalt.Length);
            Buffer.BlockCopy(buffer, 0, output, randSalt.Length, buffer.Length);
 
            return Convert.ToBase64String(output);
        }
    }
}

Here is a simple console driver application that demonstrates its usage:

Code Snippet

using System;
 
namespace SecureHashTestConsole
{
    class Program
    {
        static void Main(string[] args)
        {
            var testValues = new[] {"Testing" , "2", "4343", "help", "This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here"};
            foreach (string strTestValue in testValues)
            {
                for (int i = 0; i < 10; i++)
                {
                    String hash = SecureHashHelper.CreateSecureHash(strTestValue);
                    Console.WriteLine("Hash #{0} of '{1}' = '{2}'", i+1, strTestValue, hash);
                    Console.WriteLine("Comparison = {0} (should be True)", SecureHashHelper.CompareSecureHash(hash, strTestValue));
                    Console.WriteLine("Comparison = {0} (should be False)", SecureHashHelper.CompareSecureHash(hash, strTestValue + " "));
                }
            }
        }
    }
}

Example Output

Hash #1 of 'Testing' = 'qxj1uwwjKs+4sP9mUc96gpUIVYRcyD2FliNh'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #2 of 'Testing' = '4E7baUemZRmM7oYWk2aWdQ5jpnQ8OJcr4HmS'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #3 of 'Testing' = '0RRhLyFefkcLP9X5qWp/Nih8RNjezPzHm2hM'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #4 of 'Testing' = '/c2cdFCuc+eWAOxp192gSeSdNzWkddxkOuvD'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #5 of 'Testing' = 'OWrWE/821zy8siYjnGA+VD6UYXFRJjaW9gom'

Comparison = True (should be True)

Comparison = False (should be False)

.....................................

Hash #10 of 'help' = 'yiTptJocrqyFRpp6Hlk4/cXF04T6SQdsqqt7'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #1 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'qeIzv06CJeDXzu4p3K/uFSxcqVH45Wm6ESjo'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #2 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'xrQm1kuNrczVrdONoS8sRjddq0zq2si02hyN'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #3 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'WO7aN2lXoZgg7m0EYDnKqoRQw7mg3CSH0ori'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #4 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'CgQq75ZBm40P2IrNzaVZRBBCjjn3QdlN2Ivt'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #5 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'wXGXvdU1TBQCluwL2Kce1x4SbUU0YuFup31V'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #6 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'Xzpdr5p4RHJ1O24LdDxruL4+nw+NOp0tjrsh'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #7 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'kYFQgmaZp3VG1LjwYslKdGHZg7nnlZfLHDHB'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #8 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'nA/7Ky7xbMpAZ1OVuLF7Ku/0YarzZXpKIfA8'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #9 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'b875hD3alHDPv4yrl46GsvgGtuQOLpsECj7o'

Comparison = True (should be True)

Comparison = False (should be False)

Hash #10 of 'This is a test to see if the hash will be small even though the text is a bit bigger than the others that are listed here' = 'BJHwf4plv/t0Hpx1oi0l0VuZa9Dq0FppYI2J'

Comparison = True (should be True)

Comparison = False (should be False)

Currently rated 3.0 by 5 people

Currently 3/5 Stars.
1
2
3
4
5

Tags: security, .net, c#

ASP.Net | C#

Utility for Encrypting .Net App.Config Sections

by martin 18. July 2011 23:00

.Net ships with aspnet_regiis.exe for encrypting configuration sections in Web.Config files, but there seems to be no quick and dirty way to do this in non-web applications. Sure, if you are packaging up software into an installation package, you can add your own custom installation action to encrypt a configuration section (see The Code Project : Implementing Protected Configuration With Windows Apps). Or you can write code into your application to automatically encrypt a config section if it encounters it in an unencrypted state (see The Code Project : Encrypting the app.config File for Windows Forms Applications). If you have IIS installed on the box, you can even rename your App.config to web.config and still use aspnet_regiis (see DotNetProfessional.com: Encrypt Sections of Web.config or App.config). But what if you just want a quick way to encrypt a config section or two without all the fuss? Or, better yet, what if you forget a password or some other important secret that is already encrypted in an app.config somewhere and you need to get it back.

Here is a simple little command line app that allows you to do app.config section encryption and decryption without having to write any addtional code (The .Net framework already takes care of decrypting these values automatically for us when we use them in our code). Only this one small EXE is required and the standard MS .Net 2.0+ runtime dlls that are already installed on your system.

AppConfigSectionEncyptor.exe (7 kb)

Below are the "Usage" instructions that result from running the EXE without any parameters. "-e" is for encrypt, "-u" is for unencrypt.

C:\>AppConfigSectionEncyptor.exe
Usage: AppConfigSectionEncyptor.exe [-e|-u] [-d] file section

Examples:
        AppConfigSectionEncyptor -e c:\MyCode\MyApp.exe mySecureSection
        AppConfigSectionEncyptor -e -d c:\MyCode\MyApp.exe mySecureSection
        AppConfigSectionEncyptor -u c:\MyCode\MyApp.exe mySecureSection
Please note that 'file' is the path to the EXE, not the config file itself.
Please note that you need adaquate permissions to the keystore for this to work.

-d = use the DPAPIProtectedConfigurationProvider for encryption instead of the default RSAProtectedConfigurationProvider.

I hope that you find this useful, but as always, here is the disclaimer:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Currently rated 3.3 by 6 people

Currently 3.333333/5 Stars.
1
2
3
4
5

Tags: .net, c#, app security, security

C# | Windows

Impossible to show client side validation summary message on form validation failure using the default MS MVC 3 validation?

by martin 15. April 2011 23:05

So Microsoft has a bug in that they do not allow you to show the validation summary message in client side validation mode when you tell it to exclude the property error detail messages in the Html.ValidationSummary(true,"Please enter the required fields below.") way.

I've seen people fix this using jquery validation and unobstructive, but not with the out of the box default MicrosoftMvcValidation.js.

So there were two problems:

The first was that passing true to ValidationSummary set the replaceValidationSummary parameter in the autogenerated window.mvcClientValidationMetadata.push statement. What they should have been doing instead is simply not initializing the validationSummaryULElement.

Ok... this isn't pretty, but it works. Simply load MicrosoftMvcValidation.js in your page and then execute the code below to dynamically patch the code.

$(function () {    // this function fixes a bug.  It allows the validation summary message to be shown clientside even when the "excludePropertyErrors" bool is set to true in Html.ValidationSummary    
	var old_validate = Sys.Mvc.FormContext.prototype.validate;    
	Sys.Mvc.FormContext.prototype.validate = function () {        
		if (this.replaceValidationSummary == false) {  // only mess with this if it is turned off (MS bug)            
			this.replaceValidationSummary = true; // this is for both the debug and non debug versions            
			this._validationSummaryULElement = null;  // this is for the debug version            
			this.$8 = null;  // this is for the non debug version (does the same thing as the line above)        
		}        
		return old_validate.apply(this);   
	}
});

Currently rated 3.0 by 5 people

Currently 3/5 Stars.
1
2
3
4
5

Tags: asp.net, .net, mvc

ASP.Net | JavaScript

ASP.NET MVC Url.Content works inconsistently in HEAD tag

by martin 26. October 2010 14:13

I was trying to fix an MVC master page so that it correctly rendered urls for the referenced style sheets and JavaScript file regardless what the URL path looked like. So I ran across this StackOverflow article which told me to use Url.Content to properly resolve my relative references from the application root (not the web server root). So I did this. The all of the sudden, I started getting crazy error messages saying "A potentially dangerous Request.Path value was detected from the client (<)." It turns out that ASP.NET process things inconsistently. After reading this post and playing around, I realized that script src URLs are treated one way and link href URLs are treated a different ways when your HEAD tag has the RunAt attribute set to Server (which it does by default). I guess I'm surprised that there wasn't more about this anomaly out there. I figured that there would be a lot more folks out there using MVC that wanted to put both script and link tags in their master page so that they worked regardless of the routing path in the URL.

Currently rated 3.0 by 5 people

Currently 3/5 Stars.
1
2
3
4
5

Tags: asp.net, mvc

ASP.Net

<< Previous posts

Krolik.net

explorations in coding, ramblings, etc.

Pivoting POCO generic lists using reflection

Writing Extended ASCII codes with BinaryWriter in .Net

Using Secure Hashing safely in .Net

Utility for Encrypting .Net App.Config Sections

Impossible to show client side validation summary message on form validation failure using the default MS MVC 3 validation?

ASP.NET MVC Url.Content works inconsistently in HEAD tag

Welcome

Pages

Month Archive

Recent Comments

Krolik.net

explorations in coding, ramblings, etc.

Pivoting POCO generic lists using reflection

Writing Extended ASCII codes with BinaryWriter in .Net

Using Secure Hashing safely in .Net

Utility for Encrypting .Net App.Config Sections

Impossible to show client side validation summary message on form validation failure using the default MS MVC 3 validation?

ASP.NET MVC Url.Content works inconsistently in HEAD tag

Welcome

Tag Cloud

Pages

Month Archive

Newsletter

Thank you

Blog Roll

Recent Comments