This is a guest post by: Mark Petherbridge
Recently, the IF (IOTA Foundation) released a series of blog posts regarding their progressions and future plans for The Coordinator (the Coo). Let's take a deep dive into this and try to demystify it.
You might also be interested in checking out the How to buy IOTA guide.
What is The Coordinator?
First, lets give you a quick overview of what the coordinator is, otherwise the rest of the article might not make sense. In a nutshell the Coordinator is a centralized "Safety Mechanism" to help protect users funds. In a Tangle Network, transactions are validated by selecting transactions many times by using the random (MCMC) Markov chain Monte Carlo method . This works by selecting 100 new transactions. Meaning the coordinator calculates how many tips directly or indirectly will reach the transaction.
- If it is less than 50% the transaction is not yet validated.
- If it is more than 50%, the transaction has a fair chance to be validated.
- If it is 99% the transaction is considered validated also known as confirmed.
In it's infancy stage the Tangle Network had a small number of nodes, an attacker could rather easily create many nodes on the network and thus create many malicious transactions.
In the above situation, if we didn't use the random walk Monte Carlo method there is a big chance that malicious transactions are selected and the Tangle could theoretically succumb to an attacker deploying over a third of the network's hash-rate, referred to as a 34% attack.
"This means that IOTA in its current form does not provide any censorship resistance, since the path of the tree is centrally directed through a Coordinator node run by the IOTA Foundation" .
The security of IOTA relies on an assumption that is an intrinsic feature of all decentralized, distributed and permission-less technologies: the ability for an attacker to exert undue influence over the network is infeasible because the resources required to do so are too great. In a Blockchain, the cumulative Proof-of-Work of all the miners secures the network against attacks. 
So, to put it succinctly, The Coordinator is a means to prevent the network from malicious and fake transactions and to protect users funds.
This sounds great, why all the FUD?
The FUD (Fear Uncertainty and Doubt) spread about the Coordinator comes from the simple fact that it is a centralized system on a space that strives for decentralization.
Did you know that Bitcoin also had it’s own version of a Coordinator called: Checkpoints? 
There is also, And I will narrowly avoid this rabbit-hole, a general consensus that many people against IOTA spread FUD deliberately to discredit it's name. Why? If IOTA becomes successful in completing it's mission statement it will be the De facto choice for the IoT (Internet of Things) which brings Machine 2 Machine payments to a whole new level. This also will make many, if not all other crypto currencies superfluous to requirements. Now imagine you put all your money into one thing and then was told something else was going to do better and you would lose. Would you not get a little testy?
What are IOTA doing to remove The Coordinator?
The IOTA Foundation recently also announced on their official @iotatoken the formation of their Research Council and also a dedicated “Coordicide” team.
At the moment, the research council is made up of:
The aim of the IOTA Research Council is to work closely with their R&D (Research & Development) department to set the strategic technological direction for the foundation, provide academic oversight to IOTA research, assess and grant funding to academic partners, and oversee collaboration with academia and other research organizations.
Wait, if Bitcoin removed Checkpoints, why is IOTA Struggling?
For that we are going to have to take a little deeper dive into Blockchain and DAG technologies and how the coordinator currently helps the tangle
Bitcoin uses Blockchain Technology and IOTA Uses DAG (Directed Acylic Graph) Technology. Whilst there are a few similarities on a basic conceptual level, for example they are both Distributed Ledger Technologies and on a decentralized level both of them allow users to reach eventual consensus over the state of a ledger, under the hood they differ somewhat. Specifically their key differences in their scaling properties and their potential use cases.
Now you have a brief idea of how IOTA and Bitcoin differ, you might have guessed that the way they achieve consensus differs. Let's take Bitcoin as an example and compare that with IOTA. We will focus on two questions:
- Where should a new block connect?
- How do we determine when a transaction is considered confirmed?
Bitcoin would use the Longest-chain rule: somewhat of a misnomer, but this rule states that a new block should approve the chain with the largest accumulated PoW (Proof of Work).
Let's say that Person (called: Chris) wants to validate a spend on the blockchain. What happens to the network, is at each time step a new block is created, that new block validates all the transactions in that block, at the point where this block is created, this has to be the the longest blockchain. That's the way that Chris can validate that this was a valid spend.
Someone could try to create an alternate blockchain, so if Chris wants to spend the coin twice, what Chris would need to do is create a chain that's longer than the longest chain.
When a transaction is validated by the network, all the signatures in the coin are checked.
This is using the transfer chain, but to prevent double spending, there's also a check of this chain of blocks and the check is: The longest chain is the one that's viewed as correct.
2. Six-block rule
This rule simply determines if a block is at least six blocks deep and part of the longest chain, it can be considered confirmed.
1. MCMC tip selection
As previously mentioned, IOTA/Tangle uses something called MCMC tip selection. Basically a new transaction chooses 2 previous transactions to approve.
2. Confirmation confidence
The confirmation confidence will run the tip selection algorithm 100 times. Then it will count how many of those 100 tips approve the transaction we want, and call it A. So the confirmation confidence of our transaction is “A percent”.
Since IOTA was built with scalability in mind, they employ a voluntary and temporary different consensus mechanism for security reasons. This is what we call The Coordinator.
Every 2 minutes, a milestone transaction is issued by the IOTA Foundation. The transactions that are approved by this are automatically deemed to have 100% confidence. Using The Coordinator, the second transaction will never have been approved in the first place. This acts as a protective mechanism while the network keeps growing toward the necessary activity from adoption needed to keep the network secure in a 100% decentralized manner.
Hopefully now you can see the reasons why it's proving difficult to remove The Coordinator
The IOTA Research Council have developed a few theoretical approaches:
- Node accountability
- Improvements to the Tip Selection Algorithm
- Freedom of Choice — The ‘Stars’ Concept
Node accountability is a way to assign a behavior to a transaction. By doing this you are able to determine if the transaction should be approved or not. For example, let’s say there is double spend attempt or a ton of re-attachments we could assign a behavior of bad. But why assign a behavior at all? Why not just approve or deny the transaction? The reason for this is to create a reputation based system. This has many similarities to the object reputation systems used for peer-to-peer file-sharing  You might also notice that this is a similar type of implementation you would find with Google. They have a page ranking functionality that determines the weight and worth of the page and assigns it a value.
The Research Council and quite possibly most of the IOTA foundation have read and reviewed this as a viable option and received superb feedback from the information security community.
The development of a reputation system will also help to avoid a Proof-of-Work race, as there would be nothing to gain from issuing too many transactions — it would be impossible to double-spend, and 'spammers' would be penalized.
Tip Selection Algorithm Improvements
At the moment, the IF and the Research Council are ‘theoretically and experimentally’ investigating the various ways and algorithms related to the MCMC tip selection in a hopes to find better starting points. Their current research is yielding some good initial results and they have already developed a large scale simulation for finding the observable parameters of transactions in the tangle and some parameters which are not so easily observable, such as exit-probability-similarity. Confused? Don’t worry, this is a complex statistical theory. (Isn’t it cool that the IOTA Foundation have people this clever working for them?). [8, 9]
Freedom of Choice — The ‘Stars’ Concept
Another consideration for their improvements and eventual removal of The Coordinator is a term that they have called “Stars”. These are nodes (a way to broadcast and receive transactions to the Tangle/IOTA network). These nodes are run by well-known public entities such as governments, corporations, or individuals with a high level of trustworthiness. Just like The Coordinator issues transactions of trustworthy-ness called milestones the nodes will issue what is a called a reference transaction. This would kind of be like ‘first-approximation’ reputation system. By allowing the network to decide and approve transactions this would in essence make it decentralized.
The IOTA Foundation have released the code for their new open source version of the coordinator. The code repository can be found at: https://github.com/iotaledger/compass.
It is important to stress that removing the need for any kind of coordinator remains our top research priority for the foreseeable future. But while the network is still reliant on a coordinator, there are some far-reaching consequences to making an open source coordinator available, both positive and negative.
Whilst the IOTA Foundation have not yet successfully managed to find the correct solution to turning off The Coordinator they have put a lot of thought/time and effort into it and continue to do so. This alone shows just how serious IOTA is and will help them become one of the biggest, if not the biggest player in this market. The IOTA Foundation are also constantly expanding and adding new members to their team and have some seriously smart people now on their advisory boards, research councils and development teams.
Throughout this entire process IOTA have been open and transparent about their developments and their failures. It is clear to see that IOTA are big players who are here to stay either way
Now, after reading this, Would you not agree that the one thing that is clear about the IOTA Foundation is that they are setting themselves up to be the future choice for IoT (Internet of Things). You might want to check out the step by step guide on How to buy IOTA. Would be a shame to miss it, right?
Remember The Golden Rule
Always DYOR (Do your own research). A redditer: DeliciousShape sums this up perfectly:
When it comes to IOTA, I always urge people to focus on what the project means and how the IOTA Foundation is progressing. Forget the token, forget the hype. Look at documentation, research quotes from founders stretching back to 2014 on old BTC forums (CfB, David, Popov, Dom) and see how they followed through on their statements, read/watch what industry experts in mobility have to say, listen to what OEMs have to say about IOTA and why OEMs matter when it comes to IoT, watch the conferences at economic summits, Julie Maupin’s speeches, and research what IOTA’s collaborations with organizations such as UNOPS actually mean, etc.. 
- 1. The Tangle: an illustrated introduction
- 2. IOTA is centralized by Eric Wall
- 3. Official IOTA FAQ
- 4. Blockchain: What are checkpoints?
- 5. Experience with an Object Reputation System for Peer-to-Peer Filesharing
- 6. Wikipedia: Page rank
- 7. Coordinator. Part 3: Approaches to Coordicide
- 8. Confusion - Exit probability
- 9. Exit probabilities and optimal stochastic control
- 10. Reputation Systems for Open Collaboration*
- 11. Coordinator. Part 4: An Open Source Coordinator
- 12. Reddit Comment by: DeliciousShape