I’m working on a project that we’re trying to take a big chunk of data and generate a hash based on the data. The intention is to have a sliding window, and each time it slides the window, it XORs that data with the previous hash. Each time you get a new hash, it is compared to an array of hashes. If it’s not found in the hash table, it will add this new hash to it for comparison of later hashes.
So that’s what is supposed to happen and I was wondering what the most efficient way of going about this is. My concern is that each processor will be constantly attempting to both add to and read from the array. As a result, this will be changing all the time so if I just copy the hash table to shared memory every time I get a new hash or packet, by the time it’s finished comparing, there’s probably new data to compare to.