Musings on Tech

My name is Kyle Teague and I'm the data scientist for GetGlue. This is my blog.

Modifying Google Sparsehash for the Intel Compiler

Posted Thu, 24 Mar 2011

I recently needed a hash map/set implementation for a C++ project. No problem I thought. I went over and grabbed Google’s Sparsehash implementation and I was off and running. Things came to a screeching halt when I tried to compile my code with the Intel C++ Compiler. Usually this gives me a 30-40% speed increase for CPU-bound code, but only if it can actually compile. After a couple hours of searching and trying various other hash map implementations I realized I was going to have to fix this myself.

The problem lies in Sparsehash’s reliance on the std::tr1 namespace. This adds lots of neat things like smart pointers and regular expressions, but unfortunately g++’s implementation is not compatible with ICPC even when using the compiler flag -std=c++0x.

To remove the reliance we’re going to need to make a couple of changes and unfortunately, these changes will require you to always define your own hash function. First we’ll need to modify /usr/include/google/sparsehash/sparseconfig.h adding #ifndef __ICC around the bad parts so it looks like this:

/* Namespace for Google classes */
#define GOOGLE_NAMESPACE ::google
 
#ifndef __ICC
/* the location of the header defining hash functions */
#define HASH_FUN_H <tr1/functional>
 

/* the namespace of the hash<> function */
#define HASH_NAMESPACE std::tr1
#endif

C++

and then further down

#ifndef __ICC
/* The system-provided hash function including the namespace. */
#define SPARSEHASH_HASH HASH_NAMESPACE::hash
#endif

C++

Then in all the of the following files: sparse_hash_set, sparse_hash_map, dense_hash_set, and dense_hash_map we’ll make the following changes.

#include HASH_FUN_H // defined in config.h

C++

to

#ifdef HASH_FUN_H
#include HASH_FUN_H // defined in config.h
#endif

C++

and

class HashFcn = SPARSEHASH_HASH, // defined in sparseconfig.h

C++

to

#ifdef SPARSEHASH_HASH
class HashFcn = SPARSEHASH_HASH, // defined in sparseconfig.h
#else
class HashFcn,
#endif

C++

And there you go! Google’s efficient hash map/set implementation for the Intel Compiler. Fortunately the keys for all my hash_maps were integers so I didn’t really need a hash function, but Intel provides much faster ones in their IPP Crypto package anyways. Enjoy the performance boost!

blog comments powered by Disqus


Creative Commons License
This work by Kyle Teague is licensed under a Creative Commons Attribution 3.0 Unported License.