This is the first case of a critical dispute:
Here is my take:
IMO it is critical that Omen keeps market creation simple. So I think the solution is not to try to specify more and more rules within market creation. From a few years of experience with prediction markets I can tell you that trying to specify those rules makes the process INCREDIBLY hard and at the end of the day you still often have potential disputes. Just a random example - e.g. in this market had a source specified but for some reason the source on this day becomes not available.
Long story short, I think to keep Omen user friendly simply the most sensible interpretation of the question should be used. And this process/ assumptions should be documented in the rules. I think this is a very good suggestion in this direction here by @clesaege
Anyhow - lets have a look at how this process could be applied to the current case:
âWill there be a day with at least 1000 reported Corona death in the US in the first 14 days of July?â
There are various sources - most, including CDC and WHO (probably the âmost officialâ ones) report numbers below. However, there is a credible source that reports >1000 on 2 days.
Now the first question is:
Is the market asking:
âWill there be at least one source reporting >1000 deathâ
Or is the market asking primarily about Corona death?
I would absolutely argue that the goal of this market is to get information about the corona death/situation and not about who is reporting what.
So if the market is about actual Corona death the next question is, how to deal with the situation that multiple sources report conflicting results. Well, also here the answer seems obvious to me: you look at what are most sources saying and potentially weight them by reputation.
Under that process to me the answer is clearly âNoâ!
A last comment - the market was also trading at the end much closer to ânoâ then to âyesâ - so ânoâ seemed absolutely the expectation of most traders.
I absolutely agree with this assessment. In other words, the market is focusing on what the actual number is, and not asking whether any one source will report that number.
Note that in the new rules proposal (I encourage people to comment on those) has a rule which would have applied:
If the market doesnât mention a specific source, the most credible outcome should be reported. In order to determine the credibility of an outcome, the quantity of sources and their credibility are to be taken into account. Credibility of sources and of outcomes should be assessed according to facts, not unproven beliefs.
âWill there be a day with at least 1000 reported Corona death in the US in the first 14 days of July?â
Under the omen rules section: Default Assumptions: the market is trying to predict whether the number of COVID19 deaths is going to be 1000(or above) deaths in the first 14 days of July 2020.
(just like when there is no time given weâd assume a UTC timezone - When there is no source given weâd assume the most reputable one).
One might argue that the market is predicting whether âsomeoneâ reports more than 1000deaths, to this I say that âsomeoneâ could be âanyoneâ including my random friend posting to Facebook from his basementâŚ
Based on that assumption âMarket is predicting COVID19 deaths to be 1000(or above) deathsâ we have 4 conflicting sources 3 reporting a NO answer and 1 reporting Yes answer.
Going with the most reputable source (CDC) --> the market should resolve to NO.
Going with an average thatâs 3 NOs against 1 Yes --> the market should resolve to NO.
PS: There is more than 15eth up for grabs right now.
Most Omen markets donât specify a source and that is officially supported as per the rules and verified market docs.
StatNews is a reputable source and canât be dismissed out of hand. In fact, they were among the first to report on Coronavirus according to the New York Times.
Furthermore, the market isnât determined by the quantity of sources, and the quality of each source is debatable. The WHO, for example, is less consistent than both StatNews and the other sources provided, and it is being portrayed as an example of a source supporting the No position when in fact it undermines that position with data suggesting deaths very close (a statistically insignificant difference) to those reported by StatNews on at least one of the two occasions in which deaths were reported to meet or exceed 1000.
The bottom line is that a reputable source reported two instances of US Coronavirus deaths meeting or exceeding 1000, satisfying the criteria of a Yes resolution.
This dispute is the result of two things: a lack of a source requirement on the part of the platform and a lack of familiarity with the subject matter on the part of those who took positions in the market. One or more Omen admins appear to be taking non-neutral positions on this for reasons other than objectivity (duh) which is concerning. I support @clesaegeâs suggestion of changing the rules although I donât agree with the specific language in the current proposal.
Itâs clear that this market should resolve to âYesâ. [âŚ] the data clearly show that Coronavirus deaths in the US reached or exceeded 1000 on not just one but two days during the given time period; first, on July 7th, at 1114 deaths, and second, on July 9th, at 1000 deaths. These data are provided by StatNews and sourced from Johns Hopkins University. [âŚ] https://imgur.com/a/L5S8hnS
This argument only holds true if you ignore this argument:
If the argument is accepted that the market primarily asked about Corona death then having one reputable source giving evidence for yes is not enough to resolve to âyesâ because by the same logic obviously a credible source exists that reported ânoâ.
Given that no source is specified in the market there are only 2 sensible things to do:
a) declare the market as invalid - with the consequence that probably >80% of the current markets would be invalid.
b) look at all the sources as form a holistic few wether overall the answer should be âyesâ or ânoâ
âWould the people who bet NO have bet NO if they were aware of the StatNews reporting source on July 10th?â
My response:
You donât say an Apple phone is a bad phone because of the Chinese knockoff version of it, just like you donât say the CDC and WHO are wrong because StatNews says they are.
On the day the market would factually be decided the market traded at 85% no.
This is also a strong indication that the sources traders where expecting to be used (WHO, CDC, âŚ) showed ânoâ.
Of course that alone can not be the argument but it should be a consideration. After all, Omen will work or fail based on users having a good experience or a bad one. I think the route Augur went with declaring many markets as invalid was a significant factor in it failing to get sustainable traction.
The option to explicitly predict on âinvalidâ in v.2 will make this problem worse. It will incentives people with no interest in the overall success of the platform to buy the invalid outcome (at e.g. 3%) and spend a lot of time lobbying/ arguing for invalid. Since they only need a be success-full 1 out of 30 times they will do it and it will cost everyones time and provide no value for the platform whatsoever.
For regular users it will simply just suck because they need to be always afraid that the market will resolve as invalid. It is almost impossible to create a market where it is not possible to in some form argue why it should be invalid.
I think it is impossible to determine if there were actually a certain number of deaths on a certain day from a certain factor. The closest thing you can do is to determine if a reputable source reports a certain number of deaths on a certain day. This is likely why the markets asks about âreported Corona deathâ, because there is no way to determine actual deaths. It has to be âreportedâ.
One of the powerful characteristics of prediction markets is that it creates incentive for people to spend time and effort learning, researching and finding information that others may not have.
In the case of this market, some people may have found very valuable information that others may have not known about.
As to the question âWould the people who bet NO have bet NO if they were aware of the STAT reporting source on July 10th?!â
I think the overwhelming answer is ânoâ. So this accounts for something.
Now this raises the question of: if a person finds a relevant piece of information, should this person share this information with the community after she bets but before the market ends, or only after the market ends. If the STAT information was shared before the market ended, I donât believe the market would have ended closer to âNoâ. So, in this case and maybe others too, where the market ends is less relevant.
You say to determine the credibility of an outcome, âŚtheir credibility of sources are to be taken into account. And that to determine credibility, this should be assessed according to facts. What facts would you take into account to determine the credibility in this dispute?
The Realitio staking is getting VERY interesting, I would love to see how Kleros decides this, but I am not sure someone is willing to put up enough stake and then bet 35 ETH to send it to Arbitration.
In agreement with @koeppelmann , I do not think this market is invalid. I think there is an answer that the majority of the Kleros arbitrators would come to. Can we get it there?!
so⌠The question really is, who is willing to bet 35 ETH of value that the Kleros arbitrators will side with you?!
Lets assume the market would ask for the price of ETH without naming a source.
Now lets assume: Coinmaketcap, Coinbase and Binance would trade below the target value. But on one exchange it would trade above - also in this case the answer for me would be pretty clearly ânoâ.
In this example, if the target was $250, and CMC showed 248.00, Coinbase showed 249.90, Binance showed 249.95, and Uniswap showed 250.10, I would argue the Market would resolve âYesâ.
If a market is about a specific Source, the Market should specify that Source. Otherwise, we have to assume reputable Sources.
using your example itâs more like CMC 200.00, Coinbase 201.00, Binance 199.95, and Uniswap showed 250.10
And somehow you the Yes camp would say the price of eth is 250âŚ
In the absence of clear source, we result to the most reputable one, coming back to this market: the Center for Disease Control (CDC) is the authority on this not some uniâŚ,
The answer is no.
(This post has been edited as from the discussion above I had misread the phrasing of the market, based on the comments above.)
The market is phrased as follows: Will there be a day with at least 1000 reported Corona death in the US in the first 14 days of July?
But for the absence of a source, the market is clearly phrased focusing on the number of reported death. There is no mention of the outcome being reported by âone sourceâ or âa sourceâ or âa credible sourceâ. In fact there is no mentioning of the source anywhere, suggesting that the person posing the question was focused on the number of death, not the reporting.
As the literal interpretation of the question does not provide us with an answer regarding the source, we need interpretation.
A lot has been written on rules of interpretation or statutory construction. I realise, we are dealing here with prediction markets, not statutes, but some of those rules are useful. We do not need to reinvent the wheel here. Under common law, there are mainly three. They could help our discussion on framing a few basic rules:
Plain meaning rule (rely on the literal/plain meaning of the words, where the literal meaning provide an outcome, do not interpret meaning, even where the outcome is unjust).
Golden rule (when the words are clear, plain and unambiguous, we are bound to give effect to that meaning, irrespective of the consequences, where it provides an answer. Where however, the literal interpretation leads to an absurdity or cannot provide an answer, we may dig deeper and apply a secondary meaning and look at context)
Mischief rule (where there is ambiguity only, our role is to suppress the mischief the law is aimed at and advance the remedy).
So in statutory interpretation, the whole is a process of progressive encirclement where the interpreter starts under the general rule with (1) the ordinary meaning of the words, (2) where there is ambiguity no answer, look at their context and (3) when that does not help either, in light of the platformâs object and purpose, what should be the meaning supporting this object and purpose. (2) and (3) usually require intervention by a dispute resolution layer, as it is not black and white, but specifying that (1) should generally apply weeds out a lot of âallegedâ disputes already.
Here, relying on (1) is insufficient, we canât rely on a literal interpretation as the phrasing does not give sufficient indication for a source. Accordingly, we should go for a variant of the golden rule. Here it is important that the majority of sources reporting the number of death that are usually considered credible would lead us to a âNoâ answer. Accordingly, my suggestion here would be no.
To alleviate the disputes going forward, I suggest:
market creators should be required to provide a resolution source - this would reduce the number of disputes such as these.
as Clement already suggested, some simple Market Rules (some variant of the above statutory interpretation rules may perhaps be useful).
Timely updates are made possible thanks to the participation of users around the world and to the dedication of a team of analysts and researchers who validate data from an ever-growing list of over 5,000 sources.
The datasets are drawn from over 15 sources and can be reviewed here.
According to the facts, worldometers has a much wider reach than statnews.
Just a random other source (the primary way how people get their information) showing ânoâ.
The only way to come to the conclusion âyesâ is to interpret the question as âwill there by any source sayingâ. Lets just for fun flip the situation - lets assume all the sources that are currently reporting blow would report âaboveâ and âStatNewsâ was the only source reporting below. In this situation I would absolutely expect the answer to be âyesâ.
It it not clear to me why people argue that in both of those cases the answer should be âyesâ and thus âyesâ somehow be the preferred/default answer. This would only make sense if the question would be explicitly about âany reportâ or âat least one reportâ.
What it comes down to is this data is all over the place. None of it matches actual deaths on a day (because this is impossible).
If you look at where Google gets its data, says Source: Wikipedia.
If you go there, says source:
âCOVID-19/Coronavirus Real Time Updates With Credible Sources in US and Canada | 1Point3Acresâ. coronavirus.1point3acres.com. Retrieved 18 July 2020.
If you go there, the US Death graph shows 1,005 deaths for July 1st. (which then doesnât show on the the Google graph).
In the end, I think because a specific source was not specified, you can look at all reputable sources that report. You shouldnât look at the average or median of reputable sources.
