Vocabularies of forum-goers

[Gabu]

I'm interested in quite a few things surrounding language, and one item I find interesting is people's choice of language.  I was bored for most of yesterday, so on a complete whim, given that you can find the complete record of any user's posts, I wrote a C# program that takes in a user's ID and then tallies every single word that that user has used and how many times the user has used it, as well as producing a count of the total number of unique words that that user has said.

Essentially, how it works is it loads each webpage containing the records of the user's posts and then uses a bunch of regular expressions to strip away everything except for the posts' content, remove quotes of other users, and then strip away everything except for spaces and words.  Then it splits each post into an array of words and either increments the number of instances of the word if it's already in the list or adds it to the list if it's a new word.

It's not perfect, and I'll be trying to make it a bit better, as, currently, it smashes together words separated by hyphens, forward slashes, etc. into one giant word, and, consequently, very slightly inflates a user's total word count.  Other than that, it works perfectly fine, however.

I've tested it on myself and on the top five posters, and below are the results.  I'll list the total words used as well as the top twenty-five words used (and how many times they were used).  Given that very common words like articles, prepositions, etc. are not terribly interesting and are always at the top since you can't form many sentences without them, I'll omit those.

The results:

A18

Total words used: 15990

people - 1686
states - 1343
state - 1268
government - 841
right - 770
federal - 750
president - 707
kerry - 697
amendment - 662
congress - 661
income - 651
constitution - 607
republican - 591
support - 563
actually - 546
rights - 545
taxes - 537
money - 502
democrats - 495
point - 492
united - 488
social - 446
election - 445
party - 443
court - 430

Other notable words:

constitutional - 253
unconstitutional - 229

The constitutional + unconstitutional combination would have been in 22nd place.

Al

Total words used: 21996

people - 1024
labour - 974
party - 862
south - 813
state - 671
election - 645
actually - 642
north - 617
county - 605
probably - 532
seats - 501
right - 474
kerry - 394
areas - 383
liberal - 383
candidate - 380
voters - 366
interesting - 365
democrats - 363
tories - 352
district - 349
government - 338
problem - 332
democratic - 327
elections - 321
years - 320

Other notable words:

union - 139

StatesRights

Total words used: 15100

people - 583
right - 559
believe - 505
state - 441
kerry - 436
florida - 401
states - 382
party - 372
agree - 362
south - 323
democrats - 321
government - 292
president - 281
years - 264
support - 236
election - 209
course - 205
great - 204
history - 202
republican - 201
money - 198
southern - 191
world - 191
wrong - 190
problem - 189

Other notable words:

lincoln - 185 (28th place)

Boss Tweed

Total words used: 11582

kerry - 994
edwards - 488
election - 294
people - 285
right - 266
president - 247
state - 239
party - 219
liberal - 207
candidate - 182
states - 179
against - 165
clinton - 159
republican - 153
south - 140
reagan - 139
senate - 139
conservative - 135
mccain - 132
votes - 131
primary - 127
polls - 122
years - 122
chance - 114
voted - 113

No notable words.

Gustaf

Total words used: 15169

people - 1048
right - 719
states - 637
election - 609
party - 590
kerry - 576
agree - 446
state - 432
sweden - 360
thought - 340
forum - 326
least - 324
democrats - 298
guess - 293
republican - 292
republicans - 278
political - 264
voters - 260
close - 257
president - 249
voted - 249
conservative - 247
course - 247
liberal - 242
believe - 236

No notable words.

Gabu

Total words used: 16010

people - 1454
votes - 957
given - 607
election - 583
senate - 537
right - 488
person - 466
section - 452
amendment - 421
party - 421
voting - 413
favor - 393
debate - 387
legislation - 381
agree - 378
president - 371
simply - 362
senators - 352
believe - 349
least - 337
government - 336
states - 324
enough - 313
state - 309
kerry - 298

Other notable words:

canada - 204

---

So far, Al has the largest utilized vocabulary, and Boss Tweed has the least.  Given that Boss Tweed has less unique words used than posts, I can only assume that he's had a very healthy amount of very short posts and of repeating himself.   A18 also has less unique words used than posts.  The rest have more.

I'm not entirely sure what to divine from these results, but they're nevertheless interesting to look at.

Any comments?  If anyone would like to know about a specific word for anyone, or would like to see the entire, gigantic list of words for anyone, let me know; I'll either post it (for the first option) or email it to you (for the second option).  Additionally, if anyone would like me to do this for themselves, just say the word; I'd be more than happy to.   Please be patient, however; given that many people have hundreds of pages of posts (or in A18's case, over a thousand), it takes my program a while to process all of the information.

[12th Doctor]

Hmmm... can you do me?

[Akno21]

How come "Bush" isn't on any of those lists? I would think he would be used pretty frequently.

[Gabu]

How come "Bush" isn't on any of those lists? I would think he would be used pretty frequently.

Oops.  I think you just found a bug; I made it just knock off every word less than five letters long in the list of words used (though they are still accounted for in the total number of words used) because so many four-letter words are very common.  I never even thought about the fact that "Bush" is only four letters long.

I won't do those six again unless someone really wants me to, but I'll make "Bush" an exception to the "over four letters" rule, given that that definitely is a very valid word to keep track of.

Hmmm... can you do me?

NO I CANNOT

Oh, all right, you've talked me into it.

You're one of the ones with hundreds of pages of posts, though, so the program will be a while.  I'll post here with the results when they're in.

[Jake]

I'd appreciate if you did me.

[Gabu]

I'd appreciate if you did me.

Okay, I'll start a run for you, as well.

[12th Doctor]

Gabu: 

Thank you

[Jake]

I'd appreciate if you did me.

Okay, I'll start a run for you, as well.

Cool

[MHS2002]

Could you give me a shot? I wonder how high Potts will be on my list.

[© tweed]

That's good work Gabu.  I would like to request that you do migrendel's unique word count too.

I'll now add to my unique word count:

foosball cricket fibula crackling

[Ebowed]

I'd like to see my results, if possible.

[Akno21]

I'd like to see how mine stack up, as well.

[Colin]

I'd like to see my results Gabu. Please, pretty please.

[Gabu]

Could you give me a shot? I wonder how high Potts will be on my list.

I'd like to see my results, if possible.

I'd like to see my results Gabu. Please, pretty please.

Okay, I added you three to the queue, as well as migrendel, just for kicks (HIGH SCORE!!!!!11).

EDIT: Somehow missed Akno's post; I added him as well.

[WalterMitty]

id be curious to see my results.

i wonder how many times i used the word 'horrible'

[Gabu]

id be curious to see my results.

i wonder how many times i used the word 'horrible'

Let's find out.    Added you.

[dazzleman]

You guys are turning this into a full-time job for Gabu.

[Gabu]

You guys are turning this into a full-time job for Gabu.

Hah, no, my computer's doing all the work.  I just open a new instance of the program, enter the username and ID, and off it goes.

Wanna give it a whirl?

[dazzleman]

You guys are turning this into a full-time job for Gabu.

Hah, no, my computer's doing all the work.  I just open a new instance of the program, enter the username and ID, and off it goes.

Wanna give it a whirl?

Sure, why not, if you don't mind.  Still, I'm afraid of what might come up.

[Emsworth]

Wanna give it a whirl?

If you don't mind, I would like to do so. My predictions are:

- Constitution (or a variant thereof)
- Rights
- People

[Gabu]

Sure, why not, if you don't mind.  Still, I'm afraid of what might come up.

Well, let's find out.

Wanna give it a whirl?

If you don't mind, I would like to do so. My predictions are:

- Constitution (or a variant thereof)
- Rights
- People

Okay, I added you, too.

To everyone else: I have to go for about two hours, but I'll leave the programs running.  Probably a good thing, too, given that nine instances of it kind of use up a lot of CPU power.

[Joe Republic]

Interesting to see that "people" was the most frequently used word for all but Boss Tweed, who used it the fourth most frequently.

Gabu, could you do me?  I'd be interested to see where "Taft" features.

[WalterMitty]

i just hoe to god my most used word isnt a typo or a misspelling.

[Jake]

Sure, why not, if you don't mind.  Still, I'm afraid of what might come up.

Democrats and Liberals will be your highest I'm sure

[Joe Republic]

i just hoe to god my most used word isnt a typo or a misspelling.

I don't know about that, but I expect it won't be capitalized.