Coming Down From The Pedestal

[Limbic~Region]

All,
I grew up in a small town in Maine without a single street light and a graduating class of around 40. I was a big fish in a small pond with regards to academia. Upon graduation and entrance into the world, things didn't change a great deal. I never matriculated into college and in every government job I have had, I have always been near the top. This all changed when I discovered Open Source, specifically the Perl community.

I discovered a world in which there were many people much smarter than me. This was both exciting and scary. I was able to learn new things at an unprecedented rate but I also needed to come to terms with the fact I wasn't as elite as I thought I was. There were a number of people whose knowledge I came to recognize and respect. The sting of being corrected publicly never quite went away but it did wane. At some point however, I found myself intermittently back up on a pedestal.

No one likes to be wrong and this is greatly accentuated in the coding community. This can really get in the way of learning and helping. I almost missed an opportunity to learn a valuable lesson from Perl Mouse because I was sure that I was right. I didn't test my assumption to discover my error until japhy, a monk that I highly respect, disagreed with me. This leaves me wondering how many other opportunities I have missed to learn in similar situations.

I am also occasionally guilty of throwing my hands up when I become frustrated with someone asking for help when they just don't get it. I unrealistically expect others to learn as quickly and as easily as I do. I wonder how many people have suffered because I have been impatient, curt, or generally unwilling to help.

This is something I will likely always struggle with. The truth is that the majority of the time I am open minded enough to listen and learn as well as patient enough to help. Recognizing undesired behavior in ourselves isn't always easy, so this meditation serves as a friendly reminder to others that share this flaw.

One last parting thought. Do not let your pride get in the way of letting others learn. Don't try and conceal your mistakes and don't be afraid of learning in public. It can result in a good discussion benefiting everyone.

[dragonchild]

I am also occasionally guilty of throwing my hands up when I become frustrated with someone asking for help when they just don't get it. I unrealistically expect others to learn as quickly and as easily as I do.

I, too, have had that kind of experience. Unlike you, I actually matriculated college, and was still one of the bigger fish. (It was a little college.) I'm wondering if you and I actually learn as quickly and as easily as we think we do. Or, is it that we have chosen not to continuously challenge ourselves?

I say this because of two things:

1. Because I now work with stvn, I've been drawn into the P6 metamodel discussions. There's a lot of really really smart people on the P6L list. These guys are so smart it's scary. Yet, I can't help thinking that the smartest ones on the list are the ones who refuse to be locked into an idea, subject everything to a sniff test, and are insatiable learners. Their brilliance, if anything, is in finding relationships between unconnected pieces of knowledge. (Read up on how Larry built Perl 1.0 to see what I'm talking about. Plus, why hasn't Larry ever given a State of the Onion that was actually on programming?)
2. My wife and her parents don't have college degrees. Yet, these are the things they are able to do that I can't, even though they're just engineering or optimization problems:
   • My wife can feed and clothe a family of 7 (including an infant and a toddler) on $200/week. (optimization)
   • My father-in-law can build cars. (engineering)
   • My mother-in-law taught herself how to program and use databases (through CrystalReports) while being a secretary.

As I approach 30 in a few weeks, I'm starting to realize that what you know, while important, is less important than either what you do with it or what you did to get that knowledge. It also doesn't hurt to have a good respect for the other guy. (Read Martin Buber's I and Thou for a good grasp of the difference between a personalized Other and an objectified Other. Then, cross-reference that with http://www.pointlesswasteoftime.com/monkeysphere.html.)

[Limbic~Region]

dragonchild,
I'm wondering if you and I actually learn as quickly and as easily as we think we do. Or, is it that we have chosen not to continuously challenge ourselves?

I will try to be as objective as I can with such a subjective question. I believe that there is a great deal of truth that I generally assimilate information at an above average rate. I also believe I am guilty of creating a false reality in which I excel at everything I do. In truth, I only choose to do things which I am good at.

As I said in the original meditation, humans are notoriously bad at recognizing and changing undesireable behavior in themselves. I have spent years shattering the illusions I have created for myself and still struggle with them every day. We justify things to ourselves to protect our fragile egos.

Helping others and learning are two things that I value very highly. Judging from the recent poll, I am not the only one. Change is a long an arduous journey but it starts with a single step ;-)

[stvn]

I also believe I am guilty of creating a false reality in which I excel at everything I do.

Is it a false reality? Or is it that you are so focused on certain aspects of what you do (aspects at which you likely do excel), that sometimes you don't notice the parts in which you are deficient.

I myself tend to do this quite often, I find something I enjoy and then hyper-focus on it, sometimes to the detriment of other aspects of my $work/$play/@etc. Sure I do my best to not neglect the "not-so-fun" stuff, but it never really gets the attention it deserves. You only need to look in my leaf clogged rain gutters to see evidence of this ;)

I think though that this is a natural human tendency, and one which is probably not such a bad thing in our ever-expanding and ever more specialized world.

In truth, I only choose to do things which I am good at.

Of course you do, because there is a good chance that you enjoy doing those things which you are good at. I mean after all, if you didn't enjoy them (on some level, some "sicker" than others), then you would not have done them enough to get good at them. I see absolutely nothing wrong with this at all, in fact I think it is an admirable quality.

And from a practical business point of view, people who enjoy what they do make the best employees because the take pride in their work. In fact, my $boss considers it a requirement, more so than any type of college degree.

[dragonchild]

I believe that there is a great deal of truth that I generally assimilate information at an above average rate.

I strongly believe learning works by hooking what you don't know onto what you do know. Essentially, this is what I call the framework theory of knowledge assimilation. If you don't know algebra, you cannot possibly understand calculus. If you don't understand set theory, you cannot possibly understand relational databases. If you don't understand how emotions work, you cannot possibly understand how people work. (Sociopaths would fall under this category, along with a lot of computer programmers I know.)

The more similar the information you're trying to learn is to your current framework, the faster you'll "understand" it because it will "hook into" your current framework very quickly. So, if you're only learning about computer programming (and similar topics), as a professional computer programmer, you'll assimilate the information much faster than someone who doesn't have your credentials. Compare that with how quickly you would assimilate how to build an overhead cam or how to make traditional Japanese cuisine (a la Iron Chef).

I ran into this in highschool. I hated Biology, because it was all memorization. By that point, I had already created my framework to be rules-based, not knowledge-based. Thus, because I wasn't being presented with the rules behind the nomenclature, I was having a hard time learning the order-phylum-genus stuff. Contrast that with how quickly I picked up math, a completely rules-based system. All I had to do in math was figure out which rule went where and I could derive everything else.

We justify things to ourselves to protect our fragile egos.

I'm not so sure our egos need to be fragile. I'm discovering that ego is strongly related to fear, in particular the fear of being alone. Humans do some really stupid stuff based on that fear, usually resulting in a self-fulfilling prophecy. It's very possible to live in an ego-lite way without too much work, just by figuring out how you stack up against the fear of being alone. (Note: I'm not saying "defeat" the fear. My belief is that no matter how you choose to deal with that fear, the very fact that you're consciously dealing with it will allow you live in an ego-lite way.)

[marto]

"The only true wisdom is in knowing you know nothing." Socrates.

IMHO a very enlightening post Limbic~Region, thank you.

Martin

[chester]

But if we know nothing, then how do we know we know nothing? More importantly, if Socrates knew nothing, then why are we still quoting him? :) (Please forgive my semi-pointless semantic arguments, I'm just in the mood.) I know more Perl than my grandmother; I know less Perl than Larry Wall. I'm pretty certain of that. Where I fall in between is a harder question (likely closer to grandmother-country), but I think it's good to be realistic and objective.

So far as the original post, I don't like saying "people smarter than me"; I'd rather say "people better than me at Perl". It's unlikely that anyone is better than me at everything. No one is the best at everything. Very very few people are the best at anything. And even if you're the best at something, there's always still room to grow. I agree that it's good to keep that in mind. Talking with someone better than me at something of interest is always a wonderful experience, to me.

[Courage]

When you think you know some area, you become less opened to new knowledge in this area, so in result it appears you'll not learn new, and will know less afterwards.

The more you're in position you have not enough knowledge, the more attention you'll be paying to otherwise unnoticed details, and when having actual knowledge, multiple times increases quality of final work.

Socrates's sentence was just generalization of this phenomena.

Try this idea on your self and you will notice personal effectivity boost. (I did).

[Aristotle]

FWIW, a better translation of that quotation might be:

The only true wisdom is in knowing that what you know is nothing.

The difference is less subtle in Greek, which has several different words that are canonically translated as “nothing,” but differ in connotations. (Its vocabulary is rife with such near-synonyms, whose precise associations are nigh impossible to preserve in translation to most languages.)

[Your Mother]

I agree with chester. Wisdom is knowing you're fallible, not in assuming yourself to be a fool.

Nice OP in any case. My own home town had one stop light when I was a boy. Part of the patience equation is the online issue. I have no trouble explaining things 100 times to someone in person but online the third time is nearing the point of ALL CAPS. Ego issues change profoundly when the communications are tone-deaf/plain-text and there are no eyes looking back at you.

[BrowserUk]

Ego issues change profoundly when the communications are tone-deaf/plain-text and there are no eyes looking back at you.

I agree, though I would re-word that to

Communication issues change profoundly when the communications are monotonic words and are read without the reader benefiting from seeing the body langauge/facial expressions of the writer/speaker. They therefore tend to add the meter and tone in their minds-ear as they read.

The minds-ear is subject to the same rose-tinting/brown-staining as the minds-eye, and "the good ol' days". In particular, it carries the assumptions, perceptions, hopes and fears--and yes, ego--of the reader, regardless of what the writer thought they were writing.

One of the problems with online communities like this is that they encourage a degree of informality and the incorporation of the partisipant's humour and other personality traits into the communications. These are extremely difficult to convey with any accuracy, even by those rare, gifted wordsmiths, much less by your average non-language graduate.

Once some level of informality has become the community norm, even attempts to formalise communications in order alleviate misunderstandings can backfire in the perception of it by the reader.

[DrHyde]

I suspect Socrates was misquoted by Bill and Ted and then badly translated. If he said anything on the subject at all then I imagine he said that true wisdom lies in always remembering that you can learn from the other guy.

In the case of a retard you can kill them and learn what they taste like with chips and mushy peas.

[cbrandtbuffalo]

In addition to perlmonks, I experience the same thing at OSCON every year. I also have had the life experience of generally being in the higher percentile. At OSCON, everyone is the "smart kid" and I immediately become all too aware how little I really know. It's humbling and exhilarating at the same time. It also makes me feel good that Open Source has so many smart people pulling for it.

[pg]

We are in an interesting industry which is forever changing, so enjoy. After you become big fish in one area, you will find that you are small fish in other 100 areas ;-) We all enjoy learning.

As far as helping others goes, you help as much as you can, but there is certainly no bound between people. On one hand, there is no need for you to feel guilty if you cannot help either because you don't have the answer, or because nobody listen to your correct answer. On the other hand, nobody is really required to take your advise, whether it is right or wrong.

You have to take it easy ;-) You say whatever seems right to you, and that's the best you can do.

As far as the ego part goes, we all have it, and we don't always listen. We try to control it, but at the end , we are all human. Again, take it easy ;-)

[pboin]

I get frustrated with the *inverse* sometimes! I work with some fairly smart people, but from various disciplines. Now, I'm a pretty bright kid too, but as we all know, especially in venues like this, there is a very wide continuum of 'brightness'.

Sometimes I'll try to describe how intelligent someone is (see Re: Coming Down From The Pedestal for an example of scary-smart), and how humbled I am at how many *really* bright people there are. I mangle data for a living, and that's fine. But some of you guys are involved in designing the languages themselves! No one seems to appreciate that there is an order of magnitude between the two, and it actually frustrates me sometimes to get the false recognition of being one of the really bright ones. I find myself saying "No, no, no. You don't understand. I'm really not as smart as you seem to think I am."

So for many of you Monks, (I have many specific names in mind), thank you for sharing your knowledge and lifting me up. It's an honor and a privilege to be in your midst.

[Aristotle]

I find myself saying “No, no, no. You don’t understand. I’m really not as smart as you seem to think I am.”

You missed one connection. The people you consider smarter feel the same way about themselves! And they do for the exact same reasons you cite, too. That’s where the lack of recognised ranking you are dismayed about stems from. The smarter people are, the more they realise the many things they’re incapable of; so the less likely they are to seek laurels.

So don’t consider yourself unworthy quite yet. That there are people more capable than you does not mean that you aren’t capable yourself.

[johndageek]

This node reminds me of a story about an old cowboy (whom I tend to identify with, through no-one’s fault but my own.)

The cowboy rode into town after having herded several hundred head of cattle through a huge number of miles of seriously unfriendly rattlesnake ridden territory. As he dismounted, dust settling from his trail worn attire a city slicker (displeased about the cowboy’s appearance) hollered “Hey, Who is your superior?”.

The cowboy spit, and drew the back of his hand over his mouth before staring into the city slickers eyes and drawling “I go no superiors . . . and damned few equals.”, before walking into the bar.

[Anonymous Monk]

The cowboy rode into town after having herded several hundred head of cattle through a huge number of miles of seriously unfriendly rattlesnake ridden territory.

Pardon my ignorance, but why are the rattlesnakes relavent during a cattle drive? What rattlesnake in it's right mind sticks around when a few hundred tons of earth shaking cattle start lumbering their way towards it? If I were a rattlesnake, I'd slither the hell away from all the noise, vibrations, and commotion, which would be wise, since being stepped on by just one of the hundreds of cattle would kill me, and probably only have enough venom in my little fangs to kill one of them in return.

A cowboy with a bunch of cattle in front of it is only going to be bothered by snakes if terribly unlucky, and ends up disturbing the rattlesnake's hiding place.

My sympathy lies with the poor rattlesnake, really. He can't sleep with all the noise and bellowing, and he has to hide from the nasty cows or get squished, or eaten by a cowboy out looking for a meal and a new snakeskin belt...

[Anonymous Monk]

You must be the city slicker.

One rattlesnake can cause a stampede. In certain parts of the country there are places where large concentrations of rattlesnakes are present. Plenty enough to cause a large number of cattle to stampede/scatter -- making the cowboy's job much more difficult and deserving of praise if well done.

It is of course true that a cow usually wins a confrontation with a snake by (mostly accidently) stepping on it.

But you forget two important factors: 1) Cows have ears 2) Rattlesnakes are named after their rattles for a reason.

If you ever surprise a rattlesnake you'll find that their rattle is (assuming the snake is old enough to have a decent size) very very loud and quite good at making mamals with ears run the other way.

[perrin]

Solidarity for small town kids from Maine! My town (Freeport) was bigger than one streetlight even when I lived there, but my class was still only around 80. I'll cook up some blueberry pancakes in celebration of how far we've come.

[BrowserUk]

Don't put that pedestal into cold storage just yet.

Whilst the entropy reasoning is undoubtedly valid at the theoretical level, and your posted algorithm had problems, in the practical sense, I'm not at all sure that you were wrong.

From my further reading on entropy, my take on it is that to detect the difference between "truely random" and a very good PRNG, the analyst needs to have a large number of samples to feed into their polynomial algorithms. Further more, those samples need to be of the raw output, or a known transposition of it so that they can re-consitute the raw output.

In the case under discussion, the shuffling of a 100,000 elements, they would need either to know the raw output of the RNG and PRNG used to shuffle the elements, or they would need to know the original ordering so that they could reconstitute them.

As the usual example of when the difference between a RNG and a PRNG becomes important is in cryptology, the GoodGuy is hardly likely to provide this information freely. The task becomes one of trying to determine the difference, by inspection, from the outside, by inspecting the encoded messages. From this viewpoint, neither the raw (P)RNG sequence, nor the transposition sequence (the plain text*/pre-shuffled data) is available.

We normally talk about the random sequence transposing the plain text, but if it is the random sequence that is being sought, the other way around is equally valid

With RNGs derived from natural sources, the raw sequence is already massaged (transition mapping--0/1 bias removal) to eliminate 'skew' from the sequences. For something to be truely random, there has to be the possibility of long sequences of all 1s. From what I read here (3rd section, 3rd paragraph), of every 2-bits of naturally random data read from the radio source, 50% of them (and 75% of the random bits) are rejected as they do not contain a transition. Ie. from the pairs of bits: 00, 01, 10 & 11, only first bit of the middle two make it through to the consumer.

So the RNG source is all ready being massaged according to some theoretical assumptions about what "truely random" is. Sure, it is still possible for that 1 in gigazillions chance of 3.2M 1 bits to be generated, but it is surely even less likely?

This may happen very rarely, but who's to say that atmospheric conditions, or several thousand years of radio-active decay will not culminate in the exact circumstances where a series of 3,200,000 1-bits is produced? Say, tomorrow.

If that exact sequence of 100,000 32-bit 0xffffffff happens to be the sequence generated and detected by the guy with the polynomial algorithm for measuring entropy, then I doubt that it will conclude that the generator is a truely random source.

Ah! You say, but that is only one sample. The polynomial algorithm needs many samples.

Yes, but what if that sequence is used to encrypt 4000 100-byte messages and the BadGuy gets his hands on most of those. He might well conclude that he has enough samples to analyse. Again, I doubt that he will conclude that the 100,000 0xffffffffs are "truely random".

And finally, the advice given when using a PRNG for cryptographical purposes (surely the only place where the difference between a RNG and a very good PRNG will ever matter?), is that a bunch of (say 8) values are taken and then combined together with a hashing algorithm to produce another (128-bit) number that gets used for the encryption.

Although several hashing algorithms have been "compromised", the compromises I am aware of are all of the nature of: "We can produce a set of (numerically calculated, but otherwise meaningless) data that will (re)produce a given hash value". They don't reconstitute the original data used to generate the hash value, and there are an infinite number of messages that could produce any given hash value.

So, if you are using a good PRNG, and you follow the advice and hash several values together to produce your encryption stream, and that stream is then transposed by combining it with the plain text. The possibility that the BadGuy can work the encrypted text, back to sufficient, sequential, raw values from the PRNG, to be able to feed it to his polynomial algorithm in order to determine that it indeed came from a PRNG with insufficient entropy to truely randomise the dataset--is minimal!

(My) Conclusion: A lower-than-theoretically-required entropied PRNG could be used to shuffle 100,000 element dataset sufficiently randomly for it to be beyond the scope of practicality for the determination to be made that it was not "truely random"--outside of the open (fully disclosed) environment of the laboratory.

Upshot: For most every practical purpose a good PRNG, used with care, is "good enough", and your inate, commonsense, gut-reaction to the theoretical statement that started that discussion was spot on.

[radiantmatrix]

Thank you for writing this. I can identify with many of the things you are saying, though my issues are tangental. I grew up truly in the country: "town" was 20 miles away. When it came to computing, I was always the smartest person I knew.

On the other hand, my father could do things in his line of work (industrial materials handling) that confound me to this day, despite his inability to learn to use a computer. Because of this, I always was able to maintain a realistic idea that having talent in the computing world didn't make me "smarter" or "better" than anyone else. This kept me from becoming an egotistical prick.

I also always knew that there were people out there who were much better programmers than I, because I used software I had no idea how to write. Thus, my biggest problem was that despite being the "big fish", I always knew that it was only because my pond was so small. When I finally got a 'net connection and started to find Pascal forums (my language of choice at the time), it was thrilling: here were people I could learn from!

Unfortunately, I haven't been lucky enough to have good mentors professionally, and when I moved on to Perl I had only limited access to the 'net and so didn't find the same community. I learned what I learned partially because I am a good learner: but finding PerlMonks has done more for me in the year or so I've been here than 3 previous years of using Perl to solve real-world problems.

I look at code I wrote 6 months ago, compared to what I wrote a week ago, and it's like that old code was written by a naive moron. So I start feeling pretty good about myself. Then I see someone like merlyn post solutions to SOPW that are so elegant that they border on art, and I am reminded how little it is I really know.

I don't think it gets said often enough: thank you to all the PerlMonks for helping me get where I am, and for driving me to become ever better!

[hsmyers]

The view from on top of the pedestal is nice, but the risk of falling is great.