The Insanity of Generating All Possible GUIDs

Genesis There was once a StackOverflow question (“was” because it’s gone now) title “Fastest way in C# to iterate through ALL Guids Possible”. The premise was that the person wanted to hit a web server with a single request per GUID to determine if the server had data for it. Hilarity ensued. Suggestions Some people were legitimately trying to be helpful in this quixotic quest. Some pointed out: Floating point math mistakes: Read On →

Widow: Web Crawler Architecture

Widow In a previous post, I went over the justification for building my own web crawler named Widow. Here I will explain my alternative method for building a large-scale web crawler. Keep in mind that the crawler is still a work in progress (as of the end of 2015) so this is not final. There is still some future work to be done, which will be discussed at the end. Read On →

Goroutine IDs

Do Goroutine IDs Exist? Yes, of course. The runtime has to have some way to track them. Should I use them? No. No. No. No. Are There Packages I Can Use? There exist packages from people on the Go team, with colorful descriptions like “If you use this package, you will go straight to hell.“ There are also packages to do goroutine local storage built on top of goroutine IDs, like github. Read On →

Why I Decided to Make My Own Web Crawler

Widow The web crawler I am making is named Widow, and is freely available on GitHub. Along with Widow, there are a couple of other sub-projects that were, in my mind, necessary to have a decent crawler. It must also be able to parse the robots.txt and sitemap.xml files that websites use to direct web crawler behavior. The projects to perform both of those functions are terminator and exo. They are also available on Maven Central under the com. Read On →

Automated Website Backup to Amazon S3

After reading Lifehacker’s article linking to Gina Trapani’s article¬†about automatic website backups, I decided it would be a good idea to implement this for my own websites. Gina’s solution is great for one website, but I have multiple websites under one user. I am definitely not a bash-fu master by any stretch of the imagination so the best I could have done with bash would have been to copy Gina’s script and modify it some to fit my needs. Read On →

Function Calls, Word Alignment, and Interrupts on a TI DSP

Background I was taking the microprocessors class last fall and started writing this post… now here it is! I had a couple of labs that were giving me problems. The particular DSP I was using for my lab (the Texas Instruments TMS320F28335) has a word size of 16 bits. When writing a ton of assembly code for a couple of the labs, I came across an error that cropped up concerning word alignment that was particularly obvious when using an interrupt routine set to go off every millisecond or so. Read On →

Windows 8 Developer Preview VMWare Install

I was very excited when I heard that a Developer Preview of Windows 8 was coming out. I downloaded it (even though I went over my data cap for the month) so I could play with it in a virtual machine. I immediately went to VMWare Workstation 7.14 to attempt installing it, only to run into an error that said: MONITOR PANIC: vcpu-0:NOT_IMPLEMENTED vmcore/vmm/intr/apic.c:1903 Well, crud. That didn’t work very well. Read On →

Strange Errors, Makefiles, and Large Matrices

Strange Errors Abound The first problems I encountered numbered in the hundreds. I was getting errors that all said something like this: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before cut… After some frantic Google searching (yes, it worked this time) I came across a forum thread that contained some lists of error messages that seemed similar to mine. Someone in this thread suggested that the compiler was compiling the CUDA code as C code so the standard library of CUDA functions was not automatically being referenced. Read On →

CUDA Timing Woes

I have come to realize that programming using Nvidia’s CUDA can be a major PITA pit when a problem crops up. Generally there are few resources besides the CUDA Programming Guide that are helpful. Most of the answers that one would find in their searches are on the forums at Nvidia, and even then many searches turn up unanswered questions. My particular problem was in the realm of timing the execution of a kernel on the device. Read On →