Fail Meter: Request For Comments

Post Reply New Topic

Fail Meter

I am thinking about creating a social website. The purpose of the website is to use web2.0 techniques to crowd-source the truth. The reason I am writing this post is to get everyone's suggestions on a part of the design of the website.

Before I get your comments, please allow me to explain for a paragraph or two the parts of the website that I want to be fixed in design. These fixed ideas are the parts I am asking you to work on top of. Specifically, the part of the design I am asking for suggestions for has to do with conflict resolution procedures.

The website is modeled on the Judicial system. Plaintiff, Defendant, rules, etc... That is the gimmicky end. If you want to understand the social science angle, the website is based upon the blind taste test. The thing is, just like a blind taste test, you want volunteers who are not trolls. The truth is a big topic, so to make this work, you want as many people as possible, but they have to be able to respect the rules. That is why I ask for all website members to give the following pledges:

1. I believe in Math.
2. I will not bear false witness.
3. I believe in the Golden Rule, "do unto others as you would have them do unto you."

Anything else goes. Don't agree to all three above, then too bad, you don't get an account. These are the only bedrock rules of the site...

Once you have an account though, you can create blinded taste-test questions, of which, more later. Using the blinded versus unblinded questions with sliding scales, a whole slew of statistical analysis is possible. The main thing one can figure out is whether a person, or set of people, has more internal self-consistency in their answers than other people.

If you are more internally self-consistent it can only mean one of two things:

o Your fantasy life is preternaturally self-consistent... or
o You are closer to the truth than the other person(s).

There is the question though of what to do about people gaming the system. I expect there to be significant argument over the nature of each blinded question. Should we have panels of judges vote? Do you envision other possibilities?

First:



Second: There are many competing websites for truth. You propose to add one more.

Third: Belief proves nothing.

So … thanks, but no thanks.

Fnord:

On your first point, cartoons prove nothing, nor will my website. Allowing people to see their systemic biases is an interesting idea if you are smart and not too cynical. I have a feeling your smarts goes a lot less far than your cynicism does Fnord, but that is just my take...

Hey, no probs, but for others interested in being social about the truth, what are these website names that compete for the truth? I mean, it is 100% germane to the topic. Do they crowd-source opinion, and if not, why did you bring them up?



Let me check your previous statement using the OED.

Proof noun. - evidence or argument establishing or helping to establish a fact or the truth of a statement.
Belief noun. - an acceptance that a statement is true, or that something exists

"Belief proves nothing" is the same as the statement that:

"an acceptance that a statement is true, or that something exists, is not evidence, or an argument, helping to establish a fact, or the truth of any statement..."

Well, maybe I am just a country lawyer, but it seems like belief establishes the truth that a statement is being accepted.

Is there a silly cartoon for that, so you know, so I can off-handedly try to shut down your ideas, or do you have any real points at all Fnord to really shutdown??? My guess is you are angry and want to pass that anger on to others in the form of quick, easy, putdowns. Perfect for the internet.

I bite back though. Will you respond like an adult, or slink away? Your choice. I am trying to do something good in the world, who has time for haters in a situation like that? So maybe it would be better if you didn't respond, it could save time.

You never know though. I have to say in the past I have enjoyed your style Fnord.
You cut right to the chase...

You are going to crowd source the truth? It sounds like you are going to crowd source conformity. Conformity is not the truth.



The problem here is that many people lie or deceive.

Then nothing is gained, especially because the goal of the website is to determine a "Consensus Truth", which may not have anything to do with valid truths or the empirical evidence that supports them.

Truth by Consensus is the process of accepting statements as true simply because people generally agree upon them. Thus, is the majority of people on your website believe that "Vaccines Cause Autism", then it will be accepted as truth, regardless of the fact that no empirical evidence exists to support it.

From there, it's only a short step to "The Moon Landing Was a Hoax", "The Sandy Hook Shooting Did Not Happen", and "The Head of Elvis Presley is Being Kept Alive In a Secret White House Laboratory".

Nope, your idea is fodder for the conspiracy theorists, not for people seeking the Truth.

But don't let me talk you out of it. Go ahead and amuse yourself.

Any familiarity with kialo.com? They have an interesting way of facilitating discussion, ie. creating trees of pros and cons related to the topic where each pro or con has it's own sub-tree of pros and cons, etc. etc..

Seems like having some style restriction and author pre-approval of comments seems to help keep the riffraff out (or at least too bored by the format to want to engage).

As far as decentralized truth-seeking through, this is something that Jordan Greenhall has been talking about off and on for a long time (plenty of his Rebel Wisdom interviews) and it seems like the idea is there will be an increasingly large and connected pool of thinkers online who are seeking truth, looking for novelty in conversations, sorting through mixed-bag thinkers for bits and pieces that seem profoundly observant but perhaps not cleaned up or formatted in their proper final version yet, and these thinkers will be refining new ideas as they come to light for the sake of expediting the progress of human knowledge.

I think there are enough people who are of that mindset but I do think it has to happen organically and it might even be something as small as one person knowing four or five people who are like themselves who in turn know four or five people, etc. rather than a 'sign up here' - mainly because that has real risks of hijacking, infiltration, mismanagement, etc..

Fnord and jimmy m, honestly, you both don't know what you are talking about, but you do it with such conviction. You talk so negatively about consensus, and yet there is a reason people talk of scientific consensus, and it is not because they are being hive-minded.

Ever hear of that wisdom tale about 5 blind sages saying what an elephant is? I think both of you need to humble yourselves by thinking deeply about the meaning of that story.

There will be liars in any system, but if you make the blinds good enough, who cares?

techstepgenr8tion for the win! I will look into everything you wrote and then some. Probably good advice on the rollout too.

In our world, newspapers are in the business of selling papers. This mission can sometimes be at odds with telling the truth. The truth is many times distorted. Many normal people are being led around like sheep to the slaughter.

Most normal people have learned herd instinct. They travel with the herd. If their teachers teach them something they accept it without question. If a politician says, “the debate is over” or “there is a consensus”, they accept this as true in blind faith. They do this for convenience. But this normal approach also has a severe weakness.

IMHO, around 40% of what you read is an outright false or a misleading narrative.

On one day, the New York Times might publish an article titled Latest research indicates coffee is bad for you and drinking it will cause you to die prematurely.

The very next day another newspaper publishes an article titled Coffee is beneficial and scientist have proven that drinking it will extend your life.

So which headline is true and which is false?

The correct answer is probably both headlines are somewhat true. However, as first stated by Paracelsus, the #1 principle in toxicology is, “The dose makes the poison.” This means that below a certain dose (probably about 4 cups a day) coffee is beneficial to drink and above that threshold it can be destructive. In general, if you drink a few cups of coffee a day or not, it doesn’t really matter in extending or curtailing your life expectancy.

Consider even drinking too much water will kill you. Severe cases of hyponatremia (drinking too much water at once) can lead to water intoxication, an illness whose symptoms include headache, fatigue, nausea, vomiting, frequent urination and mental disorientation; and this condition can prove fatal. So do you really need to put a warning label on water! Or as in California, a Los Angeles Superior Court Judge just ruled that coffee must carry a warning label! Goodbye Starbucks!

Over 150 years ago Sir (Joseph) Norman Lockyer, a brilliant astrophysicist, found it very difficult to gain a voice and expand science. He created a new tool; he founded a great scientific journal called Nature and was its editor for a half century from 1869-1919. Nature encouraged controversy and vigorous debate within its pages. This was a tool he developed in gaining a voice.

But now the editors of many of these fine scientific journals have become corrupt and no longer encourage controversy and vigorous debate. They have made themselves the ultimate arbitrators. They have picked sides by deciding what is true and what is false, what to publish and what to censor. And many times they select the wrong side. They have lost their way.

Let the battles be fought out with logical arguments and research findings across the pages of the journal Nature or the many other scientific journals.

An article by the National Association of Scholars titled The Irreproducibility Crisis of Modern Science of April 2018 confirms that the crisis of reproducibility exists and compromises entire disciplines of science. In 2012 the biotechnology firm Amgen tried to reproduce 53 “landmark” studies in hematology and oncology, but could only replicate six. In that same year the director of the Center for Drug Evaluation and Research at the Food and Drug Administration estimated that up to three-quarters of published biomarker associations could not be replicated. A 2015 article in Science that presented the results of 100 replication studies of articles published in prominent psychological journals found that only 36% of the replication studies produced statistically significant results, compared with 97% of the original studies. If scientific findings are not reproducible then modern science is being built on a foundation of quicksand.

Governments run by biased politicians tend to down-select slices of funding to confirmational research at the exclusions of all others approaches. They arbitrarily pick winners and losers. As a result, many times the winners of government grants are actually awarded to defective approaches and the funding is totally wasted.

You might even wonder if the Theory of Special Relativity might not have even survived the selection process used by the scientific journal publishers of today. After all Einstein was an obscure government employee working as a clerk at the Swiss patent office.

So I agree with Norman Lockyer's approach of open debate. I can also agree with what techstepgenr9tion said about creating trees of pros and cons. But I think you also need to integrate logic fallacies into controlling the discussion. Too much of modern debate is wrapped into name calling.

A good book on the subject of logic fallacies is
The Skeptics' Guide to the Universe: How to know what's really real in a world increasingly full of fake, by Dr. Steven Novella.

Yes, we do. I deal with people every day who put more faith in consensus opinions than they do in established scientific facts.

If you tell a lie big enough and keep repeating it, people will eventually come to believe it.  The more people who believe it, the more likely it will be accepted as "truth". Eventually, something that "Everybody knows" becomes an officially-recognized "fact".  This is how the poor, women, and certain races came to be labelled as "inferior" and then subjugated, enslaved, used, abused, and disposed of by the ruling classes. This is how non-existent weapons of mass destruction were used as an excuse to start the Iraq war (which is still going on).  This is how anti-vaxxers have almost single-handedly brought back smallpox and other diseases from the brink of extinction.

And you want to encourage this process?  Or did you just request comments only so you could shoot down the ones you don't agree with and insult the people who made them?

You did request comments, remember?

jimmy m, given all the cited examples you gave, would it not be the case that ideas associated with Fail Meter could provide a counterbalance to the problems you listed. The whole point of Fail Meter is to detect systemic cognitive biases in thinking using blinded surveys (you think you like coke better than pepsi, put it to the blind taste-test!).

I mean, you list several issues that are not all the same, such as the reproducibility scandal (that scientists are vigorously addressing right now), and so on; I see nothing in what you write that gives me pause for thought about Fail Meter though, interesting writing though it may be.

Fnord, I know where you are coming from, but I think you are not giving my idea a fair evaluation. I think you are doing this because of your admittedly negative interactions with people in the past:

I deal with people every day who put more faith in consensus opinions than they do in established scientific facts.

They would put more faith in the established scientific consensus ideas if they had to experience it for themselves. There is a lot we ask the populace to take on faith for science. Hell, I have never been in space, how can I honestly know the earth is round, for instance? If you do not understand an idea for yourself, and have not experienced yourself, then how do you supposedly know?

No, I want the populace to get involved in the scientific process, even at the level of public opinion. I make no apology for it. If it is possible to crowd-source encyclopedic knowledge (wikipedia), why not try to crowd-source the verity of statements too?

I also understand your hesitance in another way Fnord, I did not give any details about the math of Fail Meter, thus allowing your imagination to run rampant and assume the worst. I find fault in your thinking though because I do not see why a social feedback loop that includes individuals engaging in rigorous debate is all that different from what we call science, but when I do it for Fail Meter, you write to me as if I am some mystic snake oil salesman, which I take serious offence to.

Quora is a truth-seeker site.

People vote on the best "truth".

Which begs the question, "Do we really need another 'Quora' site?"

To be fair, he's only joining the list

I rather like Quora myself----but I don't think it's an "authority" on anything. I have fun with it.

"Just for fun"? I can't argue with that.

Pointing out Quora as a website to consider is on point. Looked at kialo.com, they seem to make lists of pro-con arguments that get voted up. Quora votes up answers, as previously explained. If anyone has other truth sites to mention, please do. I want to consider as many truth websites as possible when considering my website's design. The kialo.com website has already helped in the sense that some part of my website will probably have to have an upvote feature

Let me explain the design of Fail Meter, then you can give it a relevant critique, if you wish. The purpose of Fail Meter is to catch people in lies. The website will not be about telling you what is true, it will be about telling you how self-consistent your answers are, as compared to other users.

Let me give an admittedly simplistic example. I am sure I bias the example, so please concentrate more on the principle of the example than on details.

Question 1: "Do you believe in the Holy Ghost?"

An atheist person would say no, but for those who say yes, a trap is in wait. The other thing to understand about Fail Meter is that you get an undifferentiated stream of questions. I hope keeping track of what you answered before so you don't get trapped would then be next to impossible.

The person who created the holy ghost question did it as a trap. Here is the trap. There are people who have no problem saying they believe in the Holy Ghost but don't believe in ghosts, let's call them semi-atheists (Dana Scully from X-Files comes to mind).

Question 2: "Do you believe the Holy Ghost is a ghost?"
Question 3: "Do you believe in ghosts?"

On a zero to 100% scale, the truth value of Q1 should be (according to basic fuzzy logic rules)

Q1 = Q2 * Q3.

So an atheistic thinking person would be fine. Q1 = 0 = Q2 * Q3 = 0 * 1. The Dana Scully semi-atheist is off though, because they would say Q1 = 1, but on the rest, let's say they are uncertain on the second question, Q2 = 0.5, but dislike idea of ghosts, Q3 = 0. The difference is now clear. The absolute errors are

Atheist: | 0 - 0 * 1 | = 0; Scully: | 1 - 0.5 * 0 | = 1.

The higher the error, the worse self-consistent your thought is. If you can think of a better example to trap people, please share with the rest of the class! In particular, I would be all ears to hear it. I need better examples...

So one would go on Fail Meter to give fuzzy logic evaluation of statements. You could also create logic traps if that is your desire. The same set of people would review the logic traps on the site for accuracy. Judgements have to do with classic logic statements, such as

If you believe the Holy Ghost exists and is a ghost, you must believe in ghosts.

The rest of the math just has to do with comparing how on average different people's answers are off from each other. Having the above fuzzy logic error values means other well-known statistical measures can be calculated as well. Standard Deviation, mean, mode, etc...

I know that people won't admit to being trapped. Flat Earthers are wrong a million times over... I mean, what does it take? Flying one of these people into outer space and then bringing them down again??? Even that will not work. So let's leave those nut jobbers out for now.

Mainly I am looking for process. Simplifications that can be made. How to allow people to vote. That kind of thing. Thanks for your consideration.