User talk:Rollinginhisgrave/How I understand reliable sources

Trying a definition

Hi @WhatamIdoing:, we discussed recently ish that it would be good if Wikipedia had a definition of reliable sources. I've been thinking about this for while and after a false start I wrote this up.

I failed to write a definition of reliable sources as I think it's a bit oxymoronic: being reliable is about consistency of trustworthiness. This contradicts the emphasis that each claim's trustworthiness needs to be evaluated afresh. I also think there are many factors to consider when determining if we can trust a claim. The consistency of a source (reliability) is only one, and not always the most relevant. I also think the emphasis in "reliable sources" is wrong. It shouldn't be evaluating if sources are trustworthy [for a claim]. It should be if a claim['s source] can be trusted. As a result of this, I've written a definition of what makes a claim able to be trusted.

My description of the process might sound a bit strange, especially the first step, but I wanted to reject the idea that you could understand this as the application of a limited set of attributes more trustworthy sources have.

I also think the treatment of what makes analysis/opinions etc good/desirable/valuable is a lot clearer. Rather than simply an opinion being "reliable" (?) it is defined as better if it is based on good reasoning and good evidence.

Anyway, I was hoping for some feedback. I was also hoping to hear your thoughts on whether it should be developed further, be understood as okay but kept as a personal understanding in my userspace, or whether I should try to get broader consensus. If you can spare the time and interest, thankyou. Rollinginhisgrave (talk | contributions) 09:14, 23 May 2025 (UTC)[reply]

I like this. Thanks for thinking about this and writing it up.

My first question is about #1: What is "this", in the question "Do I believe this is true?" Specifically, do you mean "When I look at this source, I believe that the source's contents are true" or do you mean "When I look at this source, the source makes me believe that the claim in the Wikipedia article is true"?

Consider a tweet from an accused murderer that says "I didn't kill my girlfriend".

The tweet might or might not convince you that the accused is innocent of the accusations. (Most murdered women are killed by their romantic partners, so purely as a statistical matter, he probably did.)
But the tweet probably convinces you that the claim in a Wikipedia article that says "The accused man denied killing his girlfriend" is true.

WhatamIdoing (talk) 18:21, 23 May 2025 (UTC)[reply]

"This" is not one or the other. "What Joe said was true" and "Joe said something" are discrete claims, and they need to be evaluated individually for whether we trust them. Does that make sense/do you think this needs to be clarified? Rollinginhisgrave (talk | contributions) 03:20, 24 May 2025 (UTC)[reply]

It's possible for a source to make me believe the source's contents are true; it is possible for a source to make me believe the Wikipedia claim is true. It is possible for both of these to be true, or neither of them, or only one of them.

If the goal is "Do I believe this is true?", then what happens if I believe the Wikipedia article's claim is true but the source is false? Is the source unreliable?

Consider:

Source true + article true = happiness
Source false + article false = don't use it
Source true + article false = WP:SOFIXIT
Source false + article true = ???

WhatamIdoing (talk) 00:17, 25 May 2025 (UTC)[reply]

When you say "Wikipedia claims", do you mean claims like "The accused man denied killing his girlfriend"?

Because that is also a source claim, just in this case the source is making the claim implicitly rather than explicitly. You can see this, as to evaluate whether you think the accused man denied killing his girlfriend, you need to look at the source (looking for things like: was hacked, if a representative released the statement can they be held as expressing his views etc).

I hope that actually responds to what you're saying. Rollinginhisgrave (talk | contributions) 02:41, 25 May 2025 (UTC)[reply]

I might read the accused's denial and become convinced that he is guilty. The source itself, then, I would evaluate as false for the source's claim ("I didn't do it!"). But it would be true for the Wikipedia article to say "The accused man denied killing his girlfriend". Is the false source reliable for the true claim? Or is the false source unreliable, because it's false? The answer to your question ("Do I believe this is true?") is no, I don't believe the source. WhatamIdoing (talk) 02:54, 25 May 2025 (UTC)[reply]

The source isn't true or false. It makes claims that can be true or false. One claim it explicitly makes is "I didn't do it." Another that it makes implicitly is "I am denying that I did it." I believe the first claim, I don't believe the second. Rollinginhisgrave (talk | contributions) 02:58, 25 May 2025 (UTC)[reply]

I'm pretty sure you have the first/second backwards here. WhatamIdoing (talk) 16:40, 29 May 2025 (UTC)[reply]

Yes, I did, apologies. Rollinginhisgrave (talk | contributions) 21:16, 29 May 2025 (UTC)[reply]

Maybe if you don't agree with the above, you could tell me what it would mean to verify "The accused man denied killing his girlfriend". WP:V defines verifiability as "verifiability means that people are able to check that information comes from a reliable source". Is the reliable source for "Wikipedia's claim" Wikipedia? Rollinginhisgrave (talk | contributions) 03:01, 25 May 2025 (UTC)[reply]

Mine may be more of a technical grammar question than I thought. You have a paragraph that begins: "The first, most important, is that you read the source and you read other sources talking about the subject. Then you ask: "Based on this source and other sources, do I believe this is true?""

The antecedent for the pronoun this is "source". Therefore the question becomes "Do I believe this source is true?"

But you say that sources can't be true or false; only their claims can be. That's true, I suppose, but a layer of abstraction that we usually elide in discussions over reliability. However, to expand it, the question would be "Do I believe that the claims in this source are true?"

The claim in my hypothetical source is "I didn't kill my girlfriend". So you are asking yourself, "Do I believe that the claim in the source that the author of the source didn't kill his girlfriend?" And I don't think that is the claim that you meant to evaluate for truth/falsity.

I wonder if the question might be something like "Based on this source and other sources, is this source useful for supporting statement in a Wikipedia article that would be true, relevant, and appropriate to include?" WhatamIdoing (talk) 16:44, 29 May 2025 (UTC)[reply]

Ah, my mistake. I've fixed the grammar.

I think without the abstraction we end up in cases where a source is true and false at once. With "Do I believe that the claims in this source are true?" When are you thinking this is applicable? For notability discussions?

The claim in my hypothetical... You're right, I've reworded this to be less clunky and more straightforward.

I asked a question a little while ago about the meaning of appropriate (at Wikipedia talk:Reliable sources/Archive 72#"Appropriate") and coming back to it now, I'm still confused about it. Could you give an example about a claim that is trustworthy but not appropriately sourced? I find Blueboar's comment ambiguous whether it's inappropriate because it's giving undue weight or because it's linking to content we don't want readers to be seeing (e.g. if it was doxing someone). Same goes for relevant. Rollinginhisgrave (talk | contributions) 21:16, 29 May 2025 (UTC)[reply]

I don't think that's an abstraction. I think that's an Equivocation.

I think the first job of a definition of reliability is to deal with the "reliable for" sense: What statement A, in a Wikipedia article B, would source C actually be reliable for? Notability is more like RSP's GUNREL concept.

For example, in your original formulation, "Based on this and other sources, do I believe what Harrison's Principles of Internal Medicine says about smoking causing lung cancer is true?" Or in my later one, "Based on this source and other sources, is what what Harrison's Principles of Internal Medicine useful for a supporting statement about smoking causing lung cancer, in the Wikipedia article on Lung cancer, that would be true, relevant, and appropriate to include?" WhatamIdoing (talk) 22:41, 29 May 2025 (UTC)[reply]

The "reliable for" sense is certainly the first thing to deal with. I am proposing that this framing is bad, and emphasizing claims rather than sources is better. I think this should be addressed before we try to make definitions, I'll give my reasons for why I think this is better:

The first is practical: I see editors all the time, both new and old, referring to reliable sources as a discrete concept without equivocation. There's an idea that sources can be divided into reliable and unreliable, which is fundamentally wrong.
We are looking for if we can trust a source to make a particular claim. Reliability is a bad word for this, as it has connotations of consistency. As a result, when editors evaluate claims, they do so against how the source generally handles this class of claim rather than by simply engaging a claim on its own terms.
Source has multiple meanings, including author, article, publisher so on. The consistency focus of reliability invariably leads to this being construed as publisher, or rarely the author; it is hard to speak of consistency for a single text. This is a bad approach: assessments of reliability should expand out from text to author to publisher. Instead, the publisher is the first port.
The multiple senses of source means, content analysis isn't a consideration when source is treated as meaning publisher. I see this frequently, indeed I have done it myself in the past plenty of times.

I think the ABC description of reliable for is imperfect. Because all claims need to be directly supported, B is redundant, it can simply be written as as What statement A would source B actually be reliable for [in the framing of source reliability]. In the alternative framing this would be rendered Can statement A in source B be trusted?

I don't really understand why we are considering whether information is relevant and appropriate in a guideline on/definition of reliable sources. They're just separate considerations. It should go in WP:V. Rollinginhisgrave (talk | contributions) 00:55, 30 May 2025 (UTC)[reply]

Just on the claim framing, I understand if you don't agree with it. I came to it when trying to figure out what are we actually trying to express here?, I am very open to changing my mind. I am also interested in how you would you simply describe what we're trying to do. Would your description use the notion of "reliability? Would it center sources rather than claims/statements? Rollinginhisgrave (talk | contributions) 01:17, 30 May 2025 (UTC)[reply]

I've thought about trying to divorce "reliable for" (or reliable-in-context) from "generally reliable". It's a pipe dream, but I think that "reliable for" and "generally reputable" would be clearer wording. WP:N, for example, is looking for sources that are (1) independent of the subject (2) secondary and (3) reputable, and which taken together, are (4) reliable for enough statements to write ~10 sentences (the last being called "SIGCOV").

NPOV would like articles to show the balance of POVs present in a variety of reputable sources, not reliable-in-context sources. "High-quality reliable source" is basically "reputable source". Things like the tweet from the boyfriend denying that he murdered his girlfriend are (a) low-quality sources but (b) 100% reliable for the exact statement we'd make, namely that he denied it. Things like grad school textbooks are reputable sources even if someone misuses them (and thus they're not reliable for the particular claims put in the article).

"Reliable for" has to center the source's relationship to the claims in the Wikipedia article. "Reputable", on the other hand, is more about the source's intrinsic characteristics.

The "source has multiple meanings" thing isn't really relevant. That's really about the qualities we look at when deciding whether to reject a source. You can't actually cite "Einstein" as your source. You have to cite something that Einstein wrote/said, that has been placed in a fixed form (e.g., a document). WhatamIdoing (talk) 02:58, 30 May 2025 (UTC)[reply]

I believe this is an okay distinction, I think I've seen you make it before. I'd like to think more about it. I did think about this type of thing wrt how the claim framing applies to WP:N and WP:NPOV, and I found it sufficient.

My question to you is: what is an example of a source that is (1) independent of the subject (2) secondary and (4) reliable for enough statements to write ~10 sentences, but is reliable in context? The murdering BF example is neither independent or secondary.

As a quibble, reputable is about reputation. I've seen plenty of sources accepted at AfD where reputation isn't established. Are you just trying to distinguish wording or are you making a point specific to reputable? Rollinginhisgrave (talk | contributions) 03:50, 30 May 2025 (UTC)[reply]

I'm mostly trying to distinguish the wording.

For your requested example, I think that a book pushing a particular POV would fit the requirements. Imagine, e.g., a book, published by a Big Five publisher, arguing that individual rights trumped public health concerns. One might be able to use it extensively in an article on (e.g.) Libertarian views of vaccines, but one would never want to lose sight of its biases and minority POV. The particular POV is unimportant, so long as it is a minority POV and not one person's unique creation; it could equally well be opposition to having a military, or delaying the age at which children begin formal education, or abolishing public libraries. WhatamIdoing (talk) 02:03, 17 June 2025 (UTC)[reply]