- Learning Objectives
- Introduction to IPFS
- Content Addressing
- Data Discovery & Connections
- Data Exchange
- IPFS Architecture
- UnixFS
- IPFS Shallow and Deep Dive Sessions"
- Background
  In this tutorial, you will get an introduction of IPFS Desktop, the graphical interface that you can use to interact with the files, folders, IPNS, pinning services, and more.
  Prerequisites
  In this tutorial, you will need to have installed (or updated) IPFS Desktop
  - Install IPFS Desktop
  - Kill any other nodes you may have started in the CLI with killall ipfs
  - Start IPFS Desktop on your computer
  Video: IPFS Desktop Walkthrough
  Instructions
  Explore the best workflows and capabilities with IPFS Desktop
  Install IPFS Desktop
  If you haven’t already, follow the comprehensive tutorial at at docs.ipfs.tech on installing IPFS Desktop onto your computer
  The Status Page
  The status is the top screen in the menu for IPFS desktop where you can see the basic information about your IPFS node.
  On this page:
  - Check and see how much data you have stored on IPFS
  - See how many peers you are connected to on IPFS
  - Check to see what your PeerID is
  - See which kubo version you are running
  The Files Page
  On this page, you can see all files and directories that are pinned to your ipfs node, either from uploading, or those that are pinned from peers on IPFS.
  - Try out the + Import button to add a file or folder from your local (computer’s) storage
  - Now, try pinning a file from IPFS. use the + Import button and put in the CID QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
  - Next, click the three dots to the left of the new CID that you just uploaded, click on the ‘share link’ and share it with your friend (or copy-paste it into the address bar of your own browser window)
  The Explore Page
  With this page, you can select an IPFS file or directory, and explore how this files is broken down into UnixFS files.
  - Take a look at the CID, and the multihash
  - Pick a file, and check out its size, and the number of links to that file
  The Peers Page
  The peers are the nodes in the IPFS network that you are connected to directly for sharing content. On the peers page:
  - Identify how many peers you are connected to
  - Notice the location, the types of connections you have.
  Activity: Identify your IPFS address, and connect with a friend
  - From the Status page click on the Advanced dropdown, and copy one of your addresses.
  - Send your address to a friend. from the Peers page, click the +Add connection button, and enter your friends address to connect as peers.
  The Settings Page
  There are several useful tools under the settings page of IPFS desktop. Here, you can change the Gateway you use. A Gateway allows you to access IPFS content-addressed content using the typical HTTP addressing that web2 applications use. With a gateway, you can access content on IPFS from your web browser.
  There are many gateways available to use, and you can choose a new one.
  - Go to the list of IPFS gateways at the Public Gateway Checker found at https://ipfs.github.io/public-gateway-checker/
  - Choose one of the gateways and copy it.
  - In the form box, type in https:// then paste the public gateway that you copied from the Public Gateway Checker.
  You are now using a different gateway to access the same IPFS public network! All of the same pinned data is available, you are just entering IPFS from a different entry point.
  Publishing to IPNS with IPFS Desktop
  With IPFS, when you publish a new piece of content, it has a new CID, meaning the data is immutable. This is great when you want to identify a certain piece of data, and make sure you know what you are accessing. There is, however, a use case for mutable data, such as a website.
  You want to give people one address where they can access, say, a website, but want to be able to make updates to it, such as adding an event page, or updating a typo. This is where IPNS comes in.
  IPNS allows you to tie an address in IPFS with the ’latest’ piece of data.
  To try this out, do the following steps
  - In the Settings page, under IPNS Publishing Keys, click the + Generate Key button, and choose a name for the key.
  - Create or add a folder on the Files page and add at least one file.
  - Choose one file, and click on the three buttons on the left side of it, then choose Publish to IPNS, and select the key you just created to publish with.
  ![Publish to IPNS(desktop9.png)
  - Once it is published, you can share this IPNS address with your neighbor by copy-pasting it
  - Share the IPNS address you copy-pasted with a neighbor and see if they can retrieve it!
  Now, if you were to upload a modified file with the same name to IPFS desktop, you friend could access the new, fresh, updated version of that image with the same IPNS address.
  Using Pinning Services with IPFS Desktop
  When you add files and folders to IPFS, typically they are only available for retrieval from others when your own node is up and running, unless someone else just happens to pin your file on their node from the network.
  Pinning services provide the service of pinning and maintaining the availability of files on IPFS for you, even when your own node isn’t running.
  You can sign up with one of the pinning services, and once you have, you an add the API key that is provided when you signed up to make your files permanently available on IPFS.
  Summary
  Congratulations! You have officially used IPFS to pin files, share with your friends, and even used IPNS to create mutable content in web3, you are now ready to start storing and sharing on the distributed web!
- Background
  In this tutorial you can get started by installing kubo (formerly go-ipfs), and starting & stopping a node
  Prerequisites
  - The latest version of Go (golang)
  - A shell where you can run Unix-type commands either on a Mac, or (Li)Unix machine, or Windows Powershell
    Optional: a Linux VM like Multipass
  - Install wget on the operating system (OS) you are using
  - IPFS Docs’ installation instructions
  Instructions
  Video – Getting Started with IPFS
  This video is the basic setup for kubo on a linux machine, using wget to grab the resources. You will also ipfs init a node and get it running with ipfs daemon.
  Install kubo (go-ipfs)
  Refer to the IPFS Docs’ installation instructions for the operating system you are using.
  Initialize an IPFS Node
  First you will want to verify that you have ipfs correctly installed on your machine.
  - Run the command ipfs --version to check it is installed. If you don’t get a version number, try uninstalling & installing again.
  Before you can run the ipfs daemon, you have to initialize and instance of IPFS on your machine, creating a node identity identified with peer identity: in the output of the CLI.
  - Run ipfs init in the CLI
  In the folder where you have initialized your node, you should be able to see a config file called .ipfs, which has data about your node. Run the command to list the directories as a long list (-l) where hidden (-a) files aren’t ignored
```
ls -la
➜ drwxr-xr-x   10 <folder/name>  staff          320 Sep 12 17:37 .ipfs
```
  You can change directory (cd) into the .ipfs directory to see the contents of the config file and list (-ls) the contents:
```
cd .ipfs
➜  .ipfs ls
api            blocks         config         datastore      datastore_spec keystore       repo.lock      version
```
  Start an IPFS Node
  Now that an instance had been created on your machine, you can start your IPFS node which will communicate and share data with other nodes on the network.
  - Run ipfs daemon in the CLI to start your new node.
  When you run ipfs daemon, if you get the error: lock <path>/.ipfs/repo.lock: someone else has the lock, it means there is another instance running on your machine. Use the command killall ipfs and try again
  Stop an IPFS Node
  When the IPFS node is running, any information you have pinned to that node is available, and you are able to retrieve data from the public IPFS network without a gateway.
  You may, however want to stop your node from time to time, and you can do so by pressing cntrl + c twice in a row in the same terminal it’s running in, or typing killall ipfs in another window.
- Background
  In this tutorial you can do some basic things with the public IPFS filesystem. This video tutorial provides instructions for how to install and run commands with the IPFS API using a Unix-type shell & operating system.
  Prerequisites
  - The latest version of Go (golang)
  - A shell where you can run Unix-type commands either on a Mac, or (Li)Unix machine, or Windows Powershell
  - IPFS installed on the OS you are using
  - IPFS Docs’ kubo (go-ipfs) Command-line commands
  Use the ipfs-update package to easily update to the latest version of ipfs.
  Download and install ipfs-update
  Run the command ipfs-update install v<0.00.00>.
  See more in the ipfs-update repo.
  Instructions
  In this tutorial, you can follow along to understand the basics of how you, as a user, can access files in the public IPFS network. Follow along with the examples to learn about pinning and adding files, how files from IPFS can be previewed and inspected, and learn a bit about how that data is created and stored on IPFS.
  Video – IPFS CLI Basics: Accessing Files
  Start and Stop an IPFS Node
  In the previous tutorial, you learned how to start and stop an IPFS node in the command line.
  - Run ipfs daemon in the CLI to start your new node
  When you run ipfs daemon, if you get the error: lock <path>/.ipfs/repo.lock: someone else has the lock, it means there is another instance running on your machine. Use the command killall ipfs and try again
  - You can stop your node by pressing cntrl + c twice in a row in the same terminal it’s running in, or typing killall ipfs in another window.
  See your IPFS Peers (Swarm)
  You can see your direct peers with the IPFS CLI, but with a simple command. By default, your ipfs node is seeded with a default list of trusted ‘bootstrap’ peers, which can be changed.
  - See your peers with the command ipfs swarm peers
  You should see a list of peers:
```
➜  ~ ipfs swarm peers
/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ
/ip4/115.231.82.177/tcp/4001/p2p/12D3KooWCpgcZkkRNd1FY7PwchxuLwo5dhkER8EVhzTjBgir49Gz
/ip4/119.193.8.109/udp/4001/quic/p2p/12D3KooWRdjT6WuQS1pcr6XmPeoGRH9XrxxayTV1bXFkv1WQnbd3
/ip4/149.102.159.78/udp/4001/quic/p2p/12D3KooWFKwrYNJC55UWYZqWr2Knrzwzky25LVn3BuUyaiaHwU3n
…
```
  you can examine peers by running ipfs id <CID>
  Read an IPFS File
  You can explore files on IPFS with the CLI tool as well. You can do this by installing IPFS Desktop or by using the URL for the web user interface (WebUI) at http://127.0.0.1:5001/webui which you should have seen when you ran ipfs init
```
➜  ~ ipfs daemon
Initializing daemon...
go-ipfs version: 0.11.0-67220edaa
Repo version: 11
System version: arm64/darwin
Golang version: go1.17.3
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.64.1/tcp/4001
Swarm listening on /ip4/192.168.64.1/udp/4001/quic
… output omitted
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready
```
  Grab the CID of a file on IPFS and use it to read that file in the CLI (ideally a text file).
  - Run ipfs cat <CID> to read that file
  Download an IPFS File with a CID
  To grab a file (you can find it on IPFS Desktop or the WebUI) from the public IPFS network and store it on your local machine:
  - Use the command ipfs get <CID>. You should be able to find the file in whichever directory you are currently working in with your terminal.
  - Use ls to list your files. You should see a CID starting with Qm.... or ba... in your local directory.
  Examine the file (ideally a text file) using vim:
```
➜  ~ vim QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
…
```
  Take a minute to explore a file in IPFS Desktop, or the WebUI to see different files on IPFS, and their structure.
  Add a File to your IPFS Node
  You can also take files shared on your local machine to your IPFS node, which is known as pinning. Simply use the command ipfs add directory/filename.extension. Make sure you have the correct path to the file from where you are working in the terminal.
  List Pinned Files on your Node
  Once you have started pinning files on your node, you can use the CLI to list them as well. You can choose the type of files listed, as well as how it is output in the CLI.
  - List the files on your node with ipfs pin ls --type=all
  Inspect a DAG
  Now that you understand how the IPFS CLI works, take a minute to explore how data on IPFS is split up. Visit the IPFS DAG Inspector at dag.ipfs.tech, and drop a file from your local machine (any folder, any file) and see how it is broken up into UnixFS leaves.
  Pin a File to your Node
  You have already downloaded data from IPFS, added data to IPFS from your local machine, however, there is one more way you can share and store files. You can do this by pinning files from another node. What this does is make the file available to the public IPFS network, without having to download the file to your local directory/ system.
  When you pin a file, it only exists as a part of your IPFS node, and is not a part of other filesystems on your computer. First, use the WebUI or IPFS Desktop to locate the CID of a file on the network.
  - Pin a file from the IPFS network to your local node with the command ipfs pin add <CID>
  Now if you run the command with ipfs pin ls --type=all you should see the CID of the file you just pinned in the list.
  List a Types of Pinned Files
  When you pin files, this can be done in different ways. There are direct pins, which pin just a single block and no others in relation to it. Recursive pins pin a given block and all of its children. Indirect pins are the result of a given block’s parent being pinned recursively.
  You can list the hashes of all the links an IPFS or IPNS object contains:
  - Use the command ipfs refs -r <CID> to see all the CIDs of objects that were recursively pinned from that object.
  Resources
  There are many additional tutorials and resources in the IPFS Docs you can use to explore the IPFS CLI.
  Additional Resources
  - Video Script
  - Docs
  - Get a CID of a file by starting IPFS and visiting localhost:5001/webui in your browser, then checking out the Explore section.
- Background
  When you run your IPFS node as a daemon, an HTTP RPC API is automatically exposed. Because every CLI command is available on the API, you can use this API to programmatically operate as if you were using the IPFS CLI. Note that the IPFS API is not a REST API; all endpoints are accessible by using the HTTP POST method.
  You can access the IPFS API endpoint for your local node at http://localhost:5001/api/v0/<operation>. Note that the 5001 port is used by default when you spin up your node.
  Although accessing the API directly through HTTP requests is a valid approach, there are tools available for the two main programming languages of the IPFS ecosystem: Go (Golang) and JavaScript.
  The main implementation of IPFS is kubo(formerly go-ipfs), which allows you to set up your node by spinning up a daemon application written in Go. JS-IPFS is an officially supported implementation of IPFS in JavaScript. You can take three approaches to use these implementations in your application.
  - Embedded node: if you want your application to spin up an IPFS node, then use Go-IPFS or JS-IPFS.
  - Client: if you already have a running IPFS node, then you can use a client written in Go or JS to communicate with the node. Use go-ipfs-api for Go, and js-ipfs-http-client for JS.
  - HTTP API: send HTTP requests directly from your Go or JS application through a HTTP gateway to interact with the node.
  Both JS-IPFS and js-ipfs-http-client work in the browser with some considerations noted in their READMEs.
  Connecting to an IPFS node by using the Go client
  In this exercise, you will use the Go client to interact with a running IPFS node.
  By the end of the exercise, you should be able to:
  - Connect to a running IPFS node.
  - Add a file.
  - Print the content of the file.
  - Download the file to your computer.
  - Add the file to IPNS.
  Video: IPFS API Tutorial
  Prerequisites
  - You must have Go installed. In this exercise, version 1.18 is used. If you do not have Go installed, refer to this page. If you want to install multiple versions of Go, refer to this page.
  - Clone the https://github.com/protocol/launchpad-tutorials Git repository, which contains all the sample applications used in the Launchpad program.
  - You must have an IPFS node running at the default 5001 port. Watch this video to learn how to spin up an IPFS node.
  Instructions
  - In an editor, open the ipfs-go-client folder of the launchpad-tutorials repository.
  - Examine the go.mod file, which contains the dependencies of the application. Note that the github.com/ipfs/go-ipfs-api is set as a dependency, and the rest of dependencies are indirect. To read more about indirect dependencies in Go, refer to this page.
  - Open the app/main.go file, which contains the code for this exercise. Notice that there are several not implemented methods, which you will complete throughout the exercise.
  Review the “main” Function
  - Review the func main() (on ~ line 39), which is the entry point of the application. This function calls the different functions that you will implement and handles their result for you.
  - A connection to the node is created by providing the location of the node’s API (the default port 5001).
```
sh := shell.NewShell("localhost:5001")
```
  The NewShell method returns a *shell.Shell object that exposes all the available methods to interact with the IPFS node.
  Add a File Using the IPFS API
  - Go back up to ~ line 15, and add a file that contains the Hello from Launchpad! text to IPFS in the addFile function:
```
func addFile(sh *shell.Shell, text string) (string, error) {
    return sh.Add(strings.NewReader(text))
}
```
  The Add method expects a reader, which can be generated from reading a local file or providing a string. If no errors have occurred, the CID of the added file is returned.
  Read a File Using the IPFS API
  - Next, you will read the content of the file by using the Cat method. Add in the following to the readFile function (on ~ line 19):
```
func readFile(sh *shell.Shell, cid string) (*string, error) {
    reader, err := sh.Cat(fmt.Sprintf("/ipfs/%s", cid))
    if err != nil {
        return nil, fmt.Errorf("Error reading the file: %s", err.Error())
    }
bytes, err := io.ReadAll(reader)
if err != nil {
    return nil, fmt.Errorf(&quot;Error reading bytes: %s&quot;, err.Error())
}

text := string(bytes)

return &amp;text, nil
}
```
  The readFile() function retrieves data and puts it in a buffer, converts it into a byte array, then converts that byte array into a string.
  An IPFS canonical path is passed to the Cat method. The Cat method returns a reader, so the io.ReadAll helper function is used to get the bytes of the file. Then, the bytes are cast into a string.
  Download a File Using the IPFS API
  - In the next function, downloadFile, you will download the file to your computer by using the Get method from the Shell struct. The func downloadFile returns a file using the CID:
```
func downloadFile(sh *shell.Shell, cid string) error {
    return sh.Get(cid, YourLocalPath)
}
```
  The Get method expects two parameters: the CID of the file and the local path of your computer where the file will be downloaded.
  YourLocalPath = "~/Path/to/Directory" is a constant defined at the beginning of the main.go file, where you should add the path on your local machine that you want the files downloaded to (at ~ line 13).
  Publish a File to IPNS Using the IPFS API
  To publish your file to IPNS you will need a public/private key pair, which is used as a pointer to a CID. You can read more about IPNS in the documentation.
  By default, when you install a local IPFS node, a public/private key pair called self is created.
  - In a new terminal window, list your IPFS keys.
```
> ipfs key list -l
<YOUR_PUBLIC_KEY>      self
```
  If you have not created any other key pair, only the self keypair should be listed. Copy the public key, which starts with k..., as you will need it to publish the file to IPNS.
  - The main.go file defines a YourPublicKey constant at the beginning. Include your public key in the constant.
```
const YourPublicKey = "k..."
```
  - Publish the file to IPNS by using the PublishWithDetails method in the addToIPNS function that is there already:
```
func addToIPNS(sh *shell.Shell, cid string) error {
    var lifetime time.Duration = 50 * time.Hour
    var ttl time.Duration = 0 * time.Microsecond
_, err := sh.PublishWithDetails(cid, YourPublicKey, lifetime, ttl, true)
return err
}
```
  The PublishWithDetails method expects several parameters:
  1. cid: the CID of the file that will be published to IPNS.
  2. key: the public key that will be used to publish the file. In the previous snippet, the public key constant is provided.
  3. lifetime: the time that the IPNS record will be valid. Basically, how long IPNS will keep the mapping relationship public key --> CID. By default, 24 hours.
  4. ttl: how long IPNS will cache the record.
  5. resolve: check if the given path can be resolved before publishing. By default, true.
  In the previous snippet, the record is kept in IPNS for 50 hours and there is no cache since the ttl variable is at 1 microsecond.
  Retrieve an IPNS Record Using the IPFS API
  - Use your public key to query IPNS in the resolveIPNS function that is there. The result will be the CID of the file that you published.
```
func resolveIPNS(sh *shell.Shell) (string, error) {
    return sh.Resolve(YourPublicKey)
}
```
  Run your Program
  Verify that everything works together by running the Go application.
  - Change directory with cd into the ipfs-go-client/app directory
  - Make sure you have ipfs running (ipfs daemon in a separate terminal window) then:
```
> go run .
```
  You should see output like:
```
Adding file to IPFS
File added with CID: QmNsA8eUBSbpdCVHMLa8Py5TcNoZ1D9U5GkginqktrqNF1
…output omitted…
IPNS is pointing to: /ipfs/QmNsA8eUBSbpdCVHMLa8Py5TcNoZ1D9U5GkginqktrqNF1
```
  You can see an example of the completed code here
- Quiz & Resources
- Additional Learning
- Learning Objectives
- Introduction to libp2p
- Building Blocks
- Connecting Peers
- The Distributed Hash Table
- Publish/Subscribe
- NAT Traversal
- Libp2p Shallow and Deep Dive Sessions
- Background
  In this tutorial, you will use a boilerplate Go project to set up two libp2p nodes and connect them, so you can begin to explore the capabilities of libp2p, such as data transmission, peer identification, routing, messaging, content discovery, and more.
  By the end of the exercise, you should be able to:
  - Create a new libp2p node
  - Print the node’s ID and multiaddresses
  - Configure the node at a basic level
  - Connect to another node
  Prerequisites
  - You must have Go installed. In this exercise, version 1.18 is used.
    You can also install multiple versions of Go.
  - Clone or fork the https://github.com/protocol/launchpad-tutorials Git repository, which contains all the sample applications used in the Launchpad program.
  Instructions
  - Open the /libp2p-go-simple-code folder of the launchpad-tutorials repository in an IDE of your preference. The app subfolder contains the template that you will complete.
  Video
  This video will demonstrate how you can create simple libp2p node in a Go application.
  Review the “main” Function
  In the main.go file, review the code, at about line 40. There are several functions that are called which you will implement in this tutorial. The main() function manages the flow of the program by calling different helper functions.
  Create the Source Node
  - In the createSourceNode function, create a libp2p node by using the libp2p.New() function. This method returns a host.Host interface, which you can use to manage the node.
```
func createSourceNode() host.Host {
	node, err := libp2p.New()
	if err != nil {
		panic(err)
	}
return node
}
```
  By default, the node gets an ID and listens at a random TCP port.
  Create the Target Node
  - Now, in the createTargetNode function, create a new node that listens at the 8007 TCP port. You can configure a node by passing several Option structs to the New(...) method.
```
func createTargetNode() host.Host {
	node, err := libp2p.New(
		libp2p.ListenAddrStrings(
			"/ip4/0.0.0.0/tcp/8007",
		),
	)
	if err != nil {
		panic(err)
	}
return node
}
```
  Connect the Nodes
  - So far, you have created two nodes; now, let’s connect sourceNode to targetNode. The host.Host interface contains a Connect method that you can use. The Connect method expects a peer.AddrInfo struct, which is an abstraction that represents the location of a peer. To create a peer.AddrInfo struct with the data of the node, you can use the host.InfoFromHost function.
```
func connectToTargetNode(sourceNode host.Host, targetNode host.Host) {
	targetNodeAddressInfo := host.InfoFromHost(targetNode)
err := sourceNode.Connect(context.Background(), *targetNodeAddressInfo)
if err != nil {
	panic(err)
}
}
```
  Count the Number of Peers of the Source Node
  - To verify that the connection works, you can list the peers connected to the node. In this example, you will simply count the number of peers and return that number to verify that your libp2p node is connected to peers.
```
func countSourceNodePeers(sourceNode host.Host) int {
	return len(sourceNode.Network().Peers())
}
```
  Run the Program
  - Now, test that the two nodes are connected by running the application. In your terminal, cd to the /libp2p-go-simple-node/app directory and run go run .
  You should see the following output, with the number of source node peers:
```
> go run .
-- SOURCE NODE INFORMATION --
ID: 12D3KooWCGcgrrrfDwzLmNeZ25543kYcewKxXzgDkGJGNXw1ZUf3
Multiaddresses: /ip4/192.168.0.10/tcp/63678, /ip4/127.0.0.1/tcp/63678, /ip6/::1/tcp/63681
-- TARGET NODE INFORMATION --
ID: 12D3KooWLkzhtJxcSnasfzkXGQzgKGqxGtUDpACZSXt3HM4Rn3op
Multiaddresses: /ip4/192.168.0.10/tcp/8007, /ip4/127.0.0.1/tcp/8007
Source node peers: 1
```
  You can see an example of the completed code here
  Now that you have two nodes connected and communicating, you can start to implement the many features available with libp2p. In a later tutorial, you will learn how to start a libp2p stream.
- In this tutorial, you will learn how to exchange data between two libp2p nodes by using streams (a source node will send messages to a target node).
  By the end of the exercise, you should be able to:
  - Set stream handlers for specific protocols
  - Open a new stream
  - Send messages through a stream
  - Read string data through a stream
  Video: Creating libp2p Handlers
  Prerequisites
  - You must have Go installed. In this exercise, version 1.18 is used. If you do not have Go installed, refer to this page. If you want to install multiple versions of Go, refer to this page.
  - Clone the https://github.com/protocol/launchpad-tutorials Git repository, which contains all the sample applications used in the Launchpad program.
  Instructions
  - Open the libp2p-go-handlers folder of the launchpad-tutorials repository in an IDE of your preference. The app subfolder contains the template that you will complete.
  - In the main.go file, review the code. There are several functions that you will implement in this tutorial. The main() function manages the flow of the program by calling different helper functions.
  Review the “main” Function
  - Let’s understand the flow of the program. This program creates two libp2p nodes. The flow of the program works as follows:
    Create the target node (this is handled by the runTargetNode function).
    The source node needs to know the location of the target node to establish a connection. Therefore, the runTargetNode function returns a peer.AddrInfo, containing the multiaddress and ID of the node.
    Create a source node, which receives the target node information as a parameter.
```
func main() {
	ctx, _ := context.WithCancel(context.Background())
// Create target node
info := runTargetNode()
// Create source node and provide the target node information
runSourceNode(info)

&lt;-ctx.Done()
}
```
  Implement the Target Node
  - In the runTargetNode function, add a new stream handler for the /hello/1.0.0 protocol. The stream handler of a protocol specifies what to do when a stream for that protocol is opened in the node. The SetStreamHandler method expects the ID of the protocol and the function to execute when a new message is received.
```
func runTargetNode() peer.AddrInfo {
	log.Printf("Creating target node...")
	targetNode := createNode()
	log.Printf("Target node created with ID '%s'", targetNode.ID().String())
// TO BE IMPLEMENTED: Set stream handler for the &quot;/hello/1.0.0&quot; protocol
targetNode.SetStreamHandler(&quot;/hello/1.0.0&quot;, func(s network.Stream) {
	log.Printf(&quot;/hello/1.0.0 stream created&quot;)
	err := readHelloProtocol(s)
	if err != nil {
		s.Reset()
	} else {
		s.Close()
	}
})

return *host.InfoFromHost(targetNode)
}
```
  In this tutorial, the /hello/1.0.0 protocol is used as an example. In a real world application, you can specify handlers for the custom protocols that your application uses.
  - Now, implement the readHelloProtocol function, which reads the content of the stream.
```
func readHelloProtocol(s network.Stream) error {
	// TO BE IMPLEMENTED: Read the stream and print its content
	buf := bufio.NewReader(s)
	message, err := buf.ReadString('\n')
	if err != nil {
		return err
	}
connection := s.Conn()

log.Printf(&quot;Message from '%s': %s&quot;, connection.RemotePeer().String(), message)
return nil
}
```
  In the function, a new reader is created and the content is read as a string until a line break (\n) is found. Then, the connection is used to find out which peer sent the message.
  Implement the Source Node
  - Let’s move to the runSourceNode function. The source node is created and connected to the target node by using the information provided as a parameter. Let’s open a stream for the /hello/1.0.0 protocol and send a message. The NewStream method expects a context, the ID of the peer to open the stream, and the ID of the protocol.
```
func runSourceNode(targetNodeInfo peer.AddrInfo) {
	log.Printf("Creating source node...")
	sourceNode := createNode()
	log.Printf("Source node created with ID '%s'", sourceNode.ID().String())
sourceNode.Connect(context.Background(), targetNodeInfo)

// TO BE IMPLEMENTED: Open stream and send message
stream, err := sourceNode.NewStream(context.Background(), targetNodeInfo.ID, &quot;/hello/1.0.0&quot;)
if err != nil {
	panic(err)
}

message := &quot;Hello from Launchpad!\n&quot;
log.Printf(&quot;Sending message...&quot;)
_, err = stream.Write([]byte(message))
if err != nil {
	panic(err)
}
}
```
  You can read the code like “use the target node’s connection to open a stream for the /hello/1.0.0 protocol”.
  Test the Application
  - Let’s test everything by running the Go application.
```
> go run  .
2022/09/08 12:08:02 Creating target node...
2022/09/08 12:08:02 Target node created with ID '12D3KooWF7r4SxND9Qf8uu9u3PtDduG5ET4NZf1Pw25BMqQEvXRP'
2022/09/08 12:08:02 Creating source node...
2022/09/08 12:08:02 Source node created with ID '12D3KooWMErSsjs4nPzTDjSvGcWipqyfJYxoSQJM57mTPWccEYQA'
2022/09/08 12:08:02 Sending message...
2022/09/08 12:08:02 /hello/1.0.0 stream created
2022/09/08 12:08:02 Message from '12D3KooWMErSsjs4nPzTDjSvGcWipqyfJYxoSQJM57mTPWccEYQA': Hello from Launchpad!
```
- Quiz & Resources
- Learning Objectives
- Introduction to IPLD
- Content Addressing & CIDs
- Graphs: Merkle DAGs
- IPLD & IPFS
- The Data Model & Codecs
- Background
  In this tutorial, you are going to use the IPFS command line API to experience IPLD pathing features which allow you navigate a directed acyclic graph (DAG). You will be navigating the DAG of the ipld.io website that is hosted on IPFS, thanks to a service called Fleek.co.
  Fleek.co can display a static website whose code is hosted on Github. So whenever there is a change to the ipld.io github repo, the changes get added to IPFS through Fleek and the new CID gets updated automatically in the DNS server. Therefore, as you go through the tutorial the CIDs may be different depending on when you go though it.
  Pre-Requisites
  - Install IPFS on the command line, if you have not done that yet, check out this tutorial
  - Download jq on the command line:
    Linux or WSL: sudo apt-get install jq
    Mac: brew install jq
  - Open the ipld.io website
  - Have the IPLD Gitub Repo on hand
  Video: IPLD Pathing
  Instructions
  Once you have installed kubo (formerly go-ipfs) and initiated and ipfs node, start up ipfs with the command ipfs daemon, then open a second terminal window to follow the instructions below.
  Install jq
  Jq is a tool to help the user view json text easier on the command line. You can go to stedolan.github.io/jq/download to find instructions for your operating system.
  Find the CID for ipld.io website
```
ipfs resolve /ipns/ipld.io
```
  You should see the following output in your CLI:
```
/ipfs/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg
```
  This command can help you find the root CID for any website that is hosted on IPFS through DNSLink. Fleek makes websites readily available on IPFS by auto-updating the DNSLink with the most recent CID.
  Inspect the Root
  Now that you have the root CID for the website ipld.io, you can inspect the DAG that makes up the data stored on IPFS.
```
ipfs dag get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg | jq
```
  You should see the following output in your CLI, which is a list of the CIDs below the root block:
```
{
  "Data": {
    "/": {
      "bytes": "CAE"
    }
  },
  "Links": [
    {
      "Hash": {
        "/": "QmY55i3pt9gJh4po9cdRJzZFafPAwF7V8etP9xQsnBZ8sD"
      },
      "Name": "FAQ",
      "Tsize": 15452
    },
    {
      "Hash": {
        "/": "QmVWdcKiK2mWpDXQj9DtHsrBqzA4hSLxov5juvaS4wrCZD"
      },
      "Name": "css",
      "Tsize": 85864
    },
…
…
{
"Hash": {
"/": "QmPNReXtnq8rDGJEbGGdYRxbiaZa7du1TZ782syYJodQM1"
},
"Name": "index.html",
"Tsize": 26494
},
{
"Hash": {
"/": "QmcVrQ8Bhk2S858P4sSedR9PNc8pq2zvLXpsTTPRnLZk86"
},
"Name": "x",
"Tsize": 685
}
]
}
```
  This has pulled in the next level of blocks connected to the root CID, and printed it in DAG-JSON format. We use the jq command to print the json in an easy-to-read layout.
  The original block is actually DAG-PB format which is used to construct UnixFS file data in IPFS. If you inspect the binary format of the data with ipfs block get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEq you will get illegible information, since it is raw DAG-PB bytes.
  DAG-PB has two top-level properties: Data and Links. The Data field has the actual bytes contained in a block. Each Link in DAG-PB has a name, a CID and a size. The Links contain the Hash or CID for the data in the next level below. The Hash property can also be inspected, leading you another level down (to the children) of content in that Merkle tree.
  You can see here the "Hash" or CID of the index.html page for the website.
  Load a Page With an IPFS Path
  When a client (i.e a website/a browser via an IPFS gateway) loads a root like in the following manner and doesn’t find a single page, it will look for an index.html link, which ipld.io has. In this case, the Links field is empty, but the Data field contains a lot of bytes so the client will load those in. The bytes are Base64 encoded.
```
ipfs dag get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/index.html | jq
```
  You should see the following output in your CLI:
```
{
    "Data": {
        "/": { "bytes": "CAIS0s4BPCFET0NUWVBF... "}
    },
    "Links": []
}
```
  If you want to navigate to any other block using IPFS pathing, just add the name of the link in the path, similar to what you would do in a linux filesystem.
  As an example, if you take a look at this directory in the IPLD website github repo:
  You can reach this same directory on the CLI by simply adding the filepath on the end of the root CID like so:
```
ipfs dag get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/docs/codecs/known | jq
```
  This command traverses the IPLD structure & finds the CID of the /docs/codec/known directory on Github. The output that you see should match the file structure of the Github repo above.
  Load a Page with IPLD Pathing
  DAG-PB is the default data format for IPFS and has some special properties that makes IPFS pathing possible.
  When inspecting a CID, you can both look for named “Links” or specify which node you would like to traverse by identifying the link by the number.
  To do this, with the ipfs get command you add /ipld to the CID, and then specify which ordinal Link you want to retrieve by appending /Links/<number>/Hash.
  Try the command
```
ipfs dag get /ipld/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/Links/3/Hash | jq
```
  This will retrieve the information about the 4th link listed under the root CID of QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg, which has a name of "docs".
```
{
    "Data": {
        "/": { "bytes": "CAIS0s4BPCFET0NUWVBF..." }
    },
    "Links": []
}
```
  The Links is an array field, so we use the number 3 to identify which child block we want to access. In this case, it’s going to navigate into the Hash for "docs" (since it’s at an index of 3) that will load the block with that CID. You can see that you get the same block you would with IPFS pathing referring to the same block:
  Inspect the Data in a Block
  If you add /Hash/Data to the end of a CID you are accessing certain fields in the DAG-PB data structure, it will inspecting the raw bytes of a block. In this example, we are examining the block at an index of 7, which happens to be the index.html page:
```
ipfs dag get /ipld/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/Links/7/Hash/Data | jq
```
  You should see something like the following in your CLI:
```
{
    "/": {  "bytes": "CAIS0s4BPCFET0NUWVBF... "}
}
```
  Switching Codecs
  IPLD pathing, by default, allows you to inspect data in DAG-JSON format (kubo transforms DAG-PB into DAG-JSON in the CLI), and not the raw byte array. We are going to use a command to inspect the data of the index.html page in it’s original format.
  This command uses the pathing command ipfs dag get, with the flag --output-codec=raw to change the codec you use to view the data from DAG-JSON to the RAW codec and you are able to view the bytes as they are.
```
ipfs dag get --output-codec=raw /ipld/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/Links/7/Hash/Data
��<!DOCTYPE html>
<html class="no-js" lang="en-US" prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb#">
<head>
…
<link rel="stylesheet" href="css/tachyons.min.css" />
<link href="https://fonts.googleapis.com/css?family=Open+Sans:400,600,700" rel="stylesheet"/>
<title>IPLD - The data model of the content-addressable web</title>
…
…
```
- IPLD Schemas
- Distributed Data Structures
- The CAR format
- IPLD Shallow and Deep Sessions
- Quiz & Resources

Scaling

Overview

Blockchains with smart contracts have proven to be extremely popular and useful. This has revealed the next challenge to overcome: scaling. When too many people uses these systems, the costs go up and performance can suffer. During its busiest times, transactions on Bitcoin and Ethereum can cost tens to hundreds of USD$. This isn’t a problem for high value transactions, but prevents average users from using these systems. Blockchains need to develop techniques to be able to scale to significantly more users. There are a wide variety of approaches that are being taken.

State channels
Sharding
Rollups
Improved consensus algorithms (new L1s)

State channels

State channels allow two-way communication between participants of the blockchain to take place (by locking up a part of the blockchain state). Bitcoin’s primary scaling solution is a state-channel based system called the Lightning network.

Sharding

A blockchain can scale without adding extra processing layers by a technique called “sharding”. Currently every node in a network stores the same data as every other node. Sharding is the process of reassigning data to be stored between only a sub-section of nodes. This reduces the total amount of data needed to be stored by any single node, but still allows all of the data to be available in the network. The tradeoff to this technique is the added complexity to manage the distribution of the data. It must be distributed evenly, so everyone has fair access, and it must be distributed robustly, so that even if many nodes go down, the data will still be available.

Rollups

Another scaling technique is to add layers on top of a blockchain’s base layer. The upper layers can take over much of the computation burden, reserving only the most important snapshot of data to be stored on the main chain. In this situation, the base chain is referred to as the layer 1 (L1) and the outer layers referred to as layer 2 (L2).

Rollups are a blockchain layering technique. An L2 rollup performs transaction execution outside the L1, then posts the transaction data on the L1. As transaction data is on the main chain, rollups enjoy the full security properties of the L1. Rollups are capable of handling hundreds to thousands of transactions per second.

There are two types of rollups:

Optimistic Rollups: assumes transactions are valid by default and only runs computation, via a fraud proof, in the event of a challenge
Zero-knowledge (Zk) Rollups: runs computation off-chain and submits a validity proof to the chain

“In general, my own view is that in the short term, optimistic rollups are likely to win out for general-purpose EVM computation and ZK rollups are likely to win out for simple payments, exchange and other application-specific use cases, but in the medium to long term ZK rollups will win out in all use cases as ZK-SNARK technology improves.”

– Vitalik Buterin, An Incomplete Guide to Rollups

How do Optimistic Rollups work?

Transactions sent to the rollup contracts are received on layer 2 by Sequencers Sequencers respond with a signed receipt pledging to accurately execute and order the received transaction, and are rewarded; Sequencers are also required to stake funds that will be slashed if they act maliciously.
If someone suspects fraud, they can prove it by alerting an adjudicator contract on the Ethereum mainnet, which is able to verify the validity of the results produced by the Sequencer using the OVM
The offending sequencer will be slashed, and some of the slashed funds will be rewarded to the whistleblower.
Examples of Optimistic Rollups: Optimism, Arbitrum

How do ZK-rollups work?

A ZK-rollup consists of two types of users: transactors and relayers. Transactors create their transfer and broadcast the transfer to the network. Relayers collect a large amount of transfers to create a rollup. It is the relayer’s job to generate the SNARK proof. Anyone can become a relayer so long as they have staked the required bond in the smart contract. This incentivizes the relayer not to tamper with or withhold a rollup.

SNARK proof: The SNARK proof is a hash that represents the change of the blockchain state. It compares a snapshot of the blockchain beforehand to a snapshot of the blockchain afterwards and reports only the changes as a verifiable hash to the mainnet.

Examples of ZK-Rollups: StarkWare, Filecoin

New L1s

Some developers choose to start new L1s entirely. They see the bottleneck occurring in the consensus itself and have ideas to improve the performance. Individual projects can only handle so many core contributors and decision makers, and can only try one series of ideas at a time. This, in addition to the draw of the being on a ground floor for a brand new token ecosystem, have spawned a proliferation of new blockchains with unique approaches to scaling. We will outline a few here.

Near

“An OS for an open web”

Token: NEAR

Near website

Near is an Ethereum-compatible chain with a focus on sharding. Rather than wait for the Ethereum development community to complete their vast scaling roadmap, the Near development team sought to quickly release a sharded-L1. Their team has been prolific and rapid in their deployment of production-ready features and has a quickly growing ecosystem.

Cosmos

“The internet of blockchains”

Token: ATOM

Cosmos website

Cosmos is a layer-1 toolkit. It allows developers to quickly create their own layer-1, with the unique property that all cosmos-toolkit L1s can talk to each other and exchange tokens directly with each other. Cosmos-derived chains get all the benefits of a native L1, without the difficulty of re-inventing the wheel. Scaling comes through a proliferation of networks, rather than having any one network solve all problems. Cosmos has pioneered proof-of-stake consensus, and it’s Tendermint consensus algorithm is one of the most popular.

The tradeoff with a cosmos-derived chain is that each has to provide its security independently from all other chains. One chain that has only 100 nodes may therefore be less secure than a more popular chain that has thousands of nodes.

The Cosmos-sponsored reference chain provides the ATOM token. It’s purpose is to act as a reference and to provide services that the other chains can use. It’s latest feature is the ability to share security resources. The ATOM chain can therefore use its vast node infrastructure to benefit all of the other chains in its ecosystem.

Polkadot

“Web3 interoperability”

Token: DOT

Polkadot website

Polkadot takes an approach similar to Cosmos, with another twist. Developers use its toolkit to create their own derived-chains, called para-chains. Para-chains are also able to communicate with other para-chains. The difference is that para-chains interact directly with , but these automatically derive their security

The Relay Chain: central chain of Polkadot. All validators of Polkadot are staked on the Relay Chain in DOT and validate for the Relay Chain. The main responsibility is to coordinate the system as a whole, including parachains. Other specific work is delegated to the parachains, which have different implementations and features.

Parachains: custom, sovereign blockchains that use the relay chain’s computing resources to confirm that transactions are accurate. Polkadot has 100 parachain slots. Parachains in Polkadot operate under a “shared security” model.

Parathreads: similar to parachains but with a pay-as-you-go model

Polkadot’s cousin chain is Kusama, which has nearly the same codebase as Polkadot. The network is an experimental development environment for teams who want to move fast and innovate/use as a testnet for Polkadot

Solana

“Web3 infrastructure for everyone”

Token: SOL

Solana website

Solana aims to have extremely high performance: transactions per-second in the hundreds-of-thousands. Using techniques inspired by the architecture of cell-phone radios, Solana’s consensus algorithm is able to agree on the ordering of events without having to reach consensus on every single one (which is the normal bottleneck) . Every transaction is stamped with a timestamp that is derived from a timing algorithm that every node can run independently. This consensus is called Proof of History and is one of the “8 innovations that make Solana the first web-scale blockchain” all eight of which are:

Proof of History (POH)— a clock before consensus;
Tower BFT — a PoH-optimized version of PBFT;
Turbine — a block propagation protocol;
Gulf Stream— Mempool-less transaction forwarding protocol;
Sealevel — Parallel smart contracts run-time;
Pipelining — a Transaction Processing Unit for validation optimization
Cloudbreak— Horizontally-Scaled Accounts Database; and
Archivers — Distributed ledger storage

Avalanche

“Create without limits”

Token: AVAX

Avalanche website

Avalanche takes an approach in-between Solana and Cosmos/Polkadot, developing a higher-performance consensus algorithm known as “Avalanche consensus” and utilizing separate purpose-built chains for each key service. It targets thousands of transactions per second and finality below 2 seconds.

Avalanches reaches its performance goals by using statistical sampling techniques to quickly establish consensus among a sub-set of nodes. Avalanche uses three chains, one for transactions (X-chain), one for smart contracts (C-chain), and one for side-chains (P-chain). This allows each chain to be nimble and focused. It also means it has extremely low hardware requirements to run a node, compared to other blockchains.

Filecoin

“A decentralized storage network for humanity’s most important information.”

Token: FIL

Filecoin website

Filecoin’s main purpose is to provide massive decentralized storage capabilities for the whole web. It’s main consensus algorithm is fairly slow, but it has a secondary scaling architecture called Interplanetary Consensus (IPC). This allows secondary chains to maintain a two-way communication channel with the parent, child and cousin chains, without the need for the centralizing sequencers of rollups, while allowing them to customize the consensus algorithm based on performance requirements.

Filecoin will be discussed more in subsequent sections.

Edit this page on GitHub