- Learning Objectives
- Introduction to IPFS
- Content Addressing
- Data Discovery & Connections
- Data Exchange
- IPFS Architecture
- UnixFS
- IPFS Shallow and Deep Dive Sessions"
- Background
  In this tutorial, you will get an introduction of IPFS Desktop, the graphical interface that you can use to interact with the files, folders, IPNS, pinning services, and more.
  Prerequisites
  In this tutorial, you will need to have installed (or updated) IPFS Desktop
  - Install IPFS Desktop
  - Kill any other nodes you may have started in the CLI with killall ipfs
  - Start IPFS Desktop on your computer
  Video: IPFS Desktop Walkthrough
  Instructions
  Explore the best workflows and capabilities with IPFS Desktop
  Install IPFS Desktop
  If you haven’t already, follow the comprehensive tutorial at at docs.ipfs.tech on installing IPFS Desktop onto your computer
  The Status Page
  The status is the top screen in the menu for IPFS desktop where you can see the basic information about your IPFS node.
  On this page:
  - Check and see how much data you have stored on IPFS
  - See how many peers you are connected to on IPFS
  - Check to see what your PeerID is
  - See which kubo version you are running
  The Files Page
  On this page, you can see all files and directories that are pinned to your ipfs node, either from uploading, or those that are pinned from peers on IPFS.
  - Try out the + Import button to add a file or folder from your local (computer’s) storage
  - Now, try pinning a file from IPFS. use the + Import button and put in the CID QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
  - Next, click the three dots to the left of the new CID that you just uploaded, click on the ‘share link’ and share it with your friend (or copy-paste it into the address bar of your own browser window)
  The Explore Page
  With this page, you can select an IPFS file or directory, and explore how this files is broken down into UnixFS files.
  - Take a look at the CID, and the multihash
  - Pick a file, and check out its size, and the number of links to that file
  The Peers Page
  The peers are the nodes in the IPFS network that you are connected to directly for sharing content. On the peers page:
  - Identify how many peers you are connected to
  - Notice the location, the types of connections you have.
  Activity: Identify your IPFS address, and connect with a friend
  - From the Status page click on the Advanced dropdown, and copy one of your addresses.
  - Send your address to a friend. from the Peers page, click the +Add connection button, and enter your friends address to connect as peers.
  The Settings Page
  There are several useful tools under the settings page of IPFS desktop. Here, you can change the Gateway you use. A Gateway allows you to access IPFS content-addressed content using the typical HTTP addressing that web2 applications use. With a gateway, you can access content on IPFS from your web browser.
  There are many gateways available to use, and you can choose a new one.
  - Go to the list of IPFS gateways at the Public Gateway Checker found at https://ipfs.github.io/public-gateway-checker/
  - Choose one of the gateways and copy it.
  - In the form box, type in https:// then paste the public gateway that you copied from the Public Gateway Checker.
  You are now using a different gateway to access the same IPFS public network! All of the same pinned data is available, you are just entering IPFS from a different entry point.
  Publishing to IPNS with IPFS Desktop
  With IPFS, when you publish a new piece of content, it has a new CID, meaning the data is immutable. This is great when you want to identify a certain piece of data, and make sure you know what you are accessing. There is, however, a use case for mutable data, such as a website.
  You want to give people one address where they can access, say, a website, but want to be able to make updates to it, such as adding an event page, or updating a typo. This is where IPNS comes in.
  IPNS allows you to tie an address in IPFS with the ’latest’ piece of data.
  To try this out, do the following steps
  - In the Settings page, under IPNS Publishing Keys, click the + Generate Key button, and choose a name for the key.
  - Create or add a folder on the Files page and add at least one file.
  - Choose one file, and click on the three buttons on the left side of it, then choose Publish to IPNS, and select the key you just created to publish with.
  ![Publish to IPNS(desktop9.png)
  - Once it is published, you can share this IPNS address with your neighbor by copy-pasting it
  - Share the IPNS address you copy-pasted with a neighbor and see if they can retrieve it!
  Now, if you were to upload a modified file with the same name to IPFS desktop, you friend could access the new, fresh, updated version of that image with the same IPNS address.
  Using Pinning Services with IPFS Desktop
  When you add files and folders to IPFS, typically they are only available for retrieval from others when your own node is up and running, unless someone else just happens to pin your file on their node from the network.
  Pinning services provide the service of pinning and maintaining the availability of files on IPFS for you, even when your own node isn’t running.
  You can sign up with one of the pinning services, and once you have, you an add the API key that is provided when you signed up to make your files permanently available on IPFS.
  Summary
  Congratulations! You have officially used IPFS to pin files, share with your friends, and even used IPNS to create mutable content in web3, you are now ready to start storing and sharing on the distributed web!
- Background
  In this tutorial you can get started by installing kubo (formerly go-ipfs), and starting & stopping a node
  Prerequisites
  - The latest version of Go (golang)
  - A shell where you can run Unix-type commands either on a Mac, or (Li)Unix machine, or Windows Powershell
    Optional: a Linux VM like Multipass
  - Install wget on the operating system (OS) you are using
  - IPFS Docs’ installation instructions
  Instructions
  Video – Getting Started with IPFS
  This video is the basic setup for kubo on a linux machine, using wget to grab the resources. You will also ipfs init a node and get it running with ipfs daemon.
  Install kubo (go-ipfs)
  Refer to the IPFS Docs’ installation instructions for the operating system you are using.
  Initialize an IPFS Node
  First you will want to verify that you have ipfs correctly installed on your machine.
  - Run the command ipfs --version to check it is installed. If you don’t get a version number, try uninstalling & installing again.
  Before you can run the ipfs daemon, you have to initialize and instance of IPFS on your machine, creating a node identity identified with peer identity: in the output of the CLI.
  - Run ipfs init in the CLI
  In the folder where you have initialized your node, you should be able to see a config file called .ipfs, which has data about your node. Run the command to list the directories as a long list (-l) where hidden (-a) files aren’t ignored
```
ls -la
➜ drwxr-xr-x   10 <folder/name>  staff          320 Sep 12 17:37 .ipfs
```
  You can change directory (cd) into the .ipfs directory to see the contents of the config file and list (-ls) the contents:
```
cd .ipfs
➜  .ipfs ls
api            blocks         config         datastore      datastore_spec keystore       repo.lock      version
```
  Start an IPFS Node
  Now that an instance had been created on your machine, you can start your IPFS node which will communicate and share data with other nodes on the network.
  - Run ipfs daemon in the CLI to start your new node.
  When you run ipfs daemon, if you get the error: lock <path>/.ipfs/repo.lock: someone else has the lock, it means there is another instance running on your machine. Use the command killall ipfs and try again
  Stop an IPFS Node
  When the IPFS node is running, any information you have pinned to that node is available, and you are able to retrieve data from the public IPFS network without a gateway.
  You may, however want to stop your node from time to time, and you can do so by pressing cntrl + c twice in a row in the same terminal it’s running in, or typing killall ipfs in another window.
- Background
  In this tutorial you can do some basic things with the public IPFS filesystem. This video tutorial provides instructions for how to install and run commands with the IPFS API using a Unix-type shell & operating system.
  Prerequisites
  - The latest version of Go (golang)
  - A shell where you can run Unix-type commands either on a Mac, or (Li)Unix machine, or Windows Powershell
  - IPFS installed on the OS you are using
  - IPFS Docs’ kubo (go-ipfs) Command-line commands
  Use the ipfs-update package to easily update to the latest version of ipfs.
  Download and install ipfs-update
  Run the command ipfs-update install v<0.00.00>.
  See more in the ipfs-update repo.
  Instructions
  In this tutorial, you can follow along to understand the basics of how you, as a user, can access files in the public IPFS network. Follow along with the examples to learn about pinning and adding files, how files from IPFS can be previewed and inspected, and learn a bit about how that data is created and stored on IPFS.
  Video – IPFS CLI Basics: Accessing Files
  Start and Stop an IPFS Node
  In the previous tutorial, you learned how to start and stop an IPFS node in the command line.
  - Run ipfs daemon in the CLI to start your new node
  When you run ipfs daemon, if you get the error: lock <path>/.ipfs/repo.lock: someone else has the lock, it means there is another instance running on your machine. Use the command killall ipfs and try again
  - You can stop your node by pressing cntrl + c twice in a row in the same terminal it’s running in, or typing killall ipfs in another window.
  See your IPFS Peers (Swarm)
  You can see your direct peers with the IPFS CLI, but with a simple command. By default, your ipfs node is seeded with a default list of trusted ‘bootstrap’ peers, which can be changed.
  - See your peers with the command ipfs swarm peers
  You should see a list of peers:
```
➜  ~ ipfs swarm peers
/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ
/ip4/115.231.82.177/tcp/4001/p2p/12D3KooWCpgcZkkRNd1FY7PwchxuLwo5dhkER8EVhzTjBgir49Gz
/ip4/119.193.8.109/udp/4001/quic/p2p/12D3KooWRdjT6WuQS1pcr6XmPeoGRH9XrxxayTV1bXFkv1WQnbd3
/ip4/149.102.159.78/udp/4001/quic/p2p/12D3KooWFKwrYNJC55UWYZqWr2Knrzwzky25LVn3BuUyaiaHwU3n
…
```
  you can examine peers by running ipfs id <CID>
  Read an IPFS File
  You can explore files on IPFS with the CLI tool as well. You can do this by installing IPFS Desktop or by using the URL for the web user interface (WebUI) at http://127.0.0.1:5001/webui which you should have seen when you ran ipfs init
```
➜  ~ ipfs daemon
Initializing daemon...
go-ipfs version: 0.11.0-67220edaa
Repo version: 11
System version: arm64/darwin
Golang version: go1.17.3
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.64.1/tcp/4001
Swarm listening on /ip4/192.168.64.1/udp/4001/quic
… output omitted
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready
```
  Grab the CID of a file on IPFS and use it to read that file in the CLI (ideally a text file).
  - Run ipfs cat <CID> to read that file
  Download an IPFS File with a CID
  To grab a file (you can find it on IPFS Desktop or the WebUI) from the public IPFS network and store it on your local machine:
  - Use the command ipfs get <CID>. You should be able to find the file in whichever directory you are currently working in with your terminal.
  - Use ls to list your files. You should see a CID starting with Qm.... or ba... in your local directory.
  Examine the file (ideally a text file) using vim:
```
➜  ~ vim QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
…
```
  Take a minute to explore a file in IPFS Desktop, or the WebUI to see different files on IPFS, and their structure.
  Add a File to your IPFS Node
  You can also take files shared on your local machine to your IPFS node, which is known as pinning. Simply use the command ipfs add directory/filename.extension. Make sure you have the correct path to the file from where you are working in the terminal.
  List Pinned Files on your Node
  Once you have started pinning files on your node, you can use the CLI to list them as well. You can choose the type of files listed, as well as how it is output in the CLI.
  - List the files on your node with ipfs pin ls --type=all
  Inspect a DAG
  Now that you understand how the IPFS CLI works, take a minute to explore how data on IPFS is split up. Visit the IPFS DAG Inspector at dag.ipfs.tech, and drop a file from your local machine (any folder, any file) and see how it is broken up into UnixFS leaves.
  Pin a File to your Node
  You have already downloaded data from IPFS, added data to IPFS from your local machine, however, there is one more way you can share and store files. You can do this by pinning files from another node. What this does is make the file available to the public IPFS network, without having to download the file to your local directory/ system.
  When you pin a file, it only exists as a part of your IPFS node, and is not a part of other filesystems on your computer. First, use the WebUI or IPFS Desktop to locate the CID of a file on the network.
  - Pin a file from the IPFS network to your local node with the command ipfs pin add <CID>
  Now if you run the command with ipfs pin ls --type=all you should see the CID of the file you just pinned in the list.
  List a Types of Pinned Files
  When you pin files, this can be done in different ways. There are direct pins, which pin just a single block and no others in relation to it. Recursive pins pin a given block and all of its children. Indirect pins are the result of a given block’s parent being pinned recursively.
  You can list the hashes of all the links an IPFS or IPNS object contains:
  - Use the command ipfs refs -r <CID> to see all the CIDs of objects that were recursively pinned from that object.
  Resources
  There are many additional tutorials and resources in the IPFS Docs you can use to explore the IPFS CLI.
  Additional Resources
  - Video Script
  - Docs
  - Get a CID of a file by starting IPFS and visiting localhost:5001/webui in your browser, then checking out the Explore section.
- Background
  When you run your IPFS node as a daemon, an HTTP RPC API is automatically exposed. Because every CLI command is available on the API, you can use this API to programmatically operate as if you were using the IPFS CLI. Note that the IPFS API is not a REST API; all endpoints are accessible by using the HTTP POST method.
  You can access the IPFS API endpoint for your local node at http://localhost:5001/api/v0/<operation>. Note that the 5001 port is used by default when you spin up your node.
  Although accessing the API directly through HTTP requests is a valid approach, there are tools available for the two main programming languages of the IPFS ecosystem: Go (Golang) and JavaScript.
  The main implementation of IPFS is kubo(formerly go-ipfs), which allows you to set up your node by spinning up a daemon application written in Go. JS-IPFS is an officially supported implementation of IPFS in JavaScript. You can take three approaches to use these implementations in your application.
  - Embedded node: if you want your application to spin up an IPFS node, then use Go-IPFS or JS-IPFS.
  - Client: if you already have a running IPFS node, then you can use a client written in Go or JS to communicate with the node. Use go-ipfs-api for Go, and js-ipfs-http-client for JS.
  - HTTP API: send HTTP requests directly from your Go or JS application through a HTTP gateway to interact with the node.
  Both JS-IPFS and js-ipfs-http-client work in the browser with some considerations noted in their READMEs.
  Connecting to an IPFS node by using the Go client
  In this exercise, you will use the Go client to interact with a running IPFS node.
  By the end of the exercise, you should be able to:
  - Connect to a running IPFS node.
  - Add a file.
  - Print the content of the file.
  - Download the file to your computer.
  - Add the file to IPNS.
  Video: IPFS API Tutorial
  Prerequisites
  - You must have Go installed. In this exercise, version 1.18 is used. If you do not have Go installed, refer to this page. If you want to install multiple versions of Go, refer to this page.
  - Clone the https://github.com/protocol/launchpad-tutorials Git repository, which contains all the sample applications used in the Launchpad program.
  - You must have an IPFS node running at the default 5001 port. Watch this video to learn how to spin up an IPFS node.
  Instructions
  - In an editor, open the ipfs-go-client folder of the launchpad-tutorials repository.
  - Examine the go.mod file, which contains the dependencies of the application. Note that the github.com/ipfs/go-ipfs-api is set as a dependency, and the rest of dependencies are indirect. To read more about indirect dependencies in Go, refer to this page.
  - Open the app/main.go file, which contains the code for this exercise. Notice that there are several not implemented methods, which you will complete throughout the exercise.
  Review the “main” Function
  - Review the func main() (on ~ line 39), which is the entry point of the application. This function calls the different functions that you will implement and handles their result for you.
  - A connection to the node is created by providing the location of the node’s API (the default port 5001).
```
sh := shell.NewShell("localhost:5001")
```
  The NewShell method returns a *shell.Shell object that exposes all the available methods to interact with the IPFS node.
  Add a File Using the IPFS API
  - Go back up to ~ line 15, and add a file that contains the Hello from Launchpad! text to IPFS in the addFile function:
```
func addFile(sh *shell.Shell, text string) (string, error) {
    return sh.Add(strings.NewReader(text))
}
```
  The Add method expects a reader, which can be generated from reading a local file or providing a string. If no errors have occurred, the CID of the added file is returned.
  Read a File Using the IPFS API
  - Next, you will read the content of the file by using the Cat method. Add in the following to the readFile function (on ~ line 19):
```
func readFile(sh *shell.Shell, cid string) (*string, error) {
    reader, err := sh.Cat(fmt.Sprintf("/ipfs/%s", cid))
    if err != nil {
        return nil, fmt.Errorf("Error reading the file: %s", err.Error())
    }
bytes, err := io.ReadAll(reader)
if err != nil {
    return nil, fmt.Errorf(&quot;Error reading bytes: %s&quot;, err.Error())
}

text := string(bytes)

return &amp;text, nil
}
```
  The readFile() function retrieves data and puts it in a buffer, converts it into a byte array, then converts that byte array into a string.
  An IPFS canonical path is passed to the Cat method. The Cat method returns a reader, so the io.ReadAll helper function is used to get the bytes of the file. Then, the bytes are cast into a string.
  Download a File Using the IPFS API
  - In the next function, downloadFile, you will download the file to your computer by using the Get method from the Shell struct. The func downloadFile returns a file using the CID:
```
func downloadFile(sh *shell.Shell, cid string) error {
    return sh.Get(cid, YourLocalPath)
}
```
  The Get method expects two parameters: the CID of the file and the local path of your computer where the file will be downloaded.
  YourLocalPath = "~/Path/to/Directory" is a constant defined at the beginning of the main.go file, where you should add the path on your local machine that you want the files downloaded to (at ~ line 13).
  Publish a File to IPNS Using the IPFS API
  To publish your file to IPNS you will need a public/private key pair, which is used as a pointer to a CID. You can read more about IPNS in the documentation.
  By default, when you install a local IPFS node, a public/private key pair called self is created.
  - In a new terminal window, list your IPFS keys.
```
> ipfs key list -l
<YOUR_PUBLIC_KEY>      self
```
  If you have not created any other key pair, only the self keypair should be listed. Copy the public key, which starts with k..., as you will need it to publish the file to IPNS.
  - The main.go file defines a YourPublicKey constant at the beginning. Include your public key in the constant.
```
const YourPublicKey = "k..."
```
  - Publish the file to IPNS by using the PublishWithDetails method in the addToIPNS function that is there already:
```
func addToIPNS(sh *shell.Shell, cid string) error {
    var lifetime time.Duration = 50 * time.Hour
    var ttl time.Duration = 0 * time.Microsecond
_, err := sh.PublishWithDetails(cid, YourPublicKey, lifetime, ttl, true)
return err
}
```
  The PublishWithDetails method expects several parameters:
  1. cid: the CID of the file that will be published to IPNS.
  2. key: the public key that will be used to publish the file. In the previous snippet, the public key constant is provided.
  3. lifetime: the time that the IPNS record will be valid. Basically, how long IPNS will keep the mapping relationship public key --> CID. By default, 24 hours.
  4. ttl: how long IPNS will cache the record.
  5. resolve: check if the given path can be resolved before publishing. By default, true.
  In the previous snippet, the record is kept in IPNS for 50 hours and there is no cache since the ttl variable is at 1 microsecond.
  Retrieve an IPNS Record Using the IPFS API
  - Use your public key to query IPNS in the resolveIPNS function that is there. The result will be the CID of the file that you published.
```
func resolveIPNS(sh *shell.Shell) (string, error) {
    return sh.Resolve(YourPublicKey)
}
```
  Run your Program
  Verify that everything works together by running the Go application.
  - Change directory with cd into the ipfs-go-client/app directory
  - Make sure you have ipfs running (ipfs daemon in a separate terminal window) then:
```
> go run .
```
  You should see output like:
```
Adding file to IPFS
File added with CID: QmNsA8eUBSbpdCVHMLa8Py5TcNoZ1D9U5GkginqktrqNF1
…output omitted…
IPNS is pointing to: /ipfs/QmNsA8eUBSbpdCVHMLa8Py5TcNoZ1D9U5GkginqktrqNF1
```
  You can see an example of the completed code here
- Quiz & Resources
- Additional Learning
- Learning Objectives
- Introduction to libp2p
- Building Blocks
- Connecting Peers
- The Distributed Hash Table
- Publish/Subscribe
- NAT Traversal
- Libp2p Shallow and Deep Dive Sessions
- Background
  In this tutorial, you will use a boilerplate Go project to set up two libp2p nodes and connect them, so you can begin to explore the capabilities of libp2p, such as data transmission, peer identification, routing, messaging, content discovery, and more.
  By the end of the exercise, you should be able to:
  - Create a new libp2p node
  - Print the node’s ID and multiaddresses
  - Configure the node at a basic level
  - Connect to another node
  Prerequisites
  - You must have Go installed. In this exercise, version 1.18 is used.
    You can also install multiple versions of Go.
  - Clone or fork the https://github.com/protocol/launchpad-tutorials Git repository, which contains all the sample applications used in the Launchpad program.
  Instructions
  - Open the /libp2p-go-simple-code folder of the launchpad-tutorials repository in an IDE of your preference. The app subfolder contains the template that you will complete.
  Video
  This video will demonstrate how you can create simple libp2p node in a Go application.
  Review the “main” Function
  In the main.go file, review the code, at about line 40. There are several functions that are called which you will implement in this tutorial. The main() function manages the flow of the program by calling different helper functions.
  Create the Source Node
  - In the createSourceNode function, create a libp2p node by using the libp2p.New() function. This method returns a host.Host interface, which you can use to manage the node.
```
func createSourceNode() host.Host {
	node, err := libp2p.New()
	if err != nil {
		panic(err)
	}
return node
}
```
  By default, the node gets an ID and listens at a random TCP port.
  Create the Target Node
  - Now, in the createTargetNode function, create a new node that listens at the 8007 TCP port. You can configure a node by passing several Option structs to the New(...) method.
```
func createTargetNode() host.Host {
	node, err := libp2p.New(
		libp2p.ListenAddrStrings(
			"/ip4/0.0.0.0/tcp/8007",
		),
	)
	if err != nil {
		panic(err)
	}
return node
}
```
  Connect the Nodes
  - So far, you have created two nodes; now, let’s connect sourceNode to targetNode. The host.Host interface contains a Connect method that you can use. The Connect method expects a peer.AddrInfo struct, which is an abstraction that represents the location of a peer. To create a peer.AddrInfo struct with the data of the node, you can use the host.InfoFromHost function.
```
func connectToTargetNode(sourceNode host.Host, targetNode host.Host) {
	targetNodeAddressInfo := host.InfoFromHost(targetNode)
err := sourceNode.Connect(context.Background(), *targetNodeAddressInfo)
if err != nil {
	panic(err)
}
}
```
  Count the Number of Peers of the Source Node
  - To verify that the connection works, you can list the peers connected to the node. In this example, you will simply count the number of peers and return that number to verify that your libp2p node is connected to peers.
```
func countSourceNodePeers(sourceNode host.Host) int {
	return len(sourceNode.Network().Peers())
}
```
  Run the Program
  - Now, test that the two nodes are connected by running the application. In your terminal, cd to the /libp2p-go-simple-node/app directory and run go run .
  You should see the following output, with the number of source node peers:
```
> go run .
-- SOURCE NODE INFORMATION --
ID: 12D3KooWCGcgrrrfDwzLmNeZ25543kYcewKxXzgDkGJGNXw1ZUf3
Multiaddresses: /ip4/192.168.0.10/tcp/63678, /ip4/127.0.0.1/tcp/63678, /ip6/::1/tcp/63681
-- TARGET NODE INFORMATION --
ID: 12D3KooWLkzhtJxcSnasfzkXGQzgKGqxGtUDpACZSXt3HM4Rn3op
Multiaddresses: /ip4/192.168.0.10/tcp/8007, /ip4/127.0.0.1/tcp/8007
Source node peers: 1
```
  You can see an example of the completed code here
  Now that you have two nodes connected and communicating, you can start to implement the many features available with libp2p. In a later tutorial, you will learn how to start a libp2p stream.
- In this tutorial, you will learn how to exchange data between two libp2p nodes by using streams (a source node will send messages to a target node).
  By the end of the exercise, you should be able to:
  - Set stream handlers for specific protocols
  - Open a new stream
  - Send messages through a stream
  - Read string data through a stream
  Video: Creating libp2p Handlers
  Prerequisites
  - You must have Go installed. In this exercise, version 1.18 is used. If you do not have Go installed, refer to this page. If you want to install multiple versions of Go, refer to this page.
  - Clone the https://github.com/protocol/launchpad-tutorials Git repository, which contains all the sample applications used in the Launchpad program.
  Instructions
  - Open the libp2p-go-handlers folder of the launchpad-tutorials repository in an IDE of your preference. The app subfolder contains the template that you will complete.
  - In the main.go file, review the code. There are several functions that you will implement in this tutorial. The main() function manages the flow of the program by calling different helper functions.
  Review the “main” Function
  - Let’s understand the flow of the program. This program creates two libp2p nodes. The flow of the program works as follows:
    Create the target node (this is handled by the runTargetNode function).
    The source node needs to know the location of the target node to establish a connection. Therefore, the runTargetNode function returns a peer.AddrInfo, containing the multiaddress and ID of the node.
    Create a source node, which receives the target node information as a parameter.
```
func main() {
	ctx, _ := context.WithCancel(context.Background())
// Create target node
info := runTargetNode()
// Create source node and provide the target node information
runSourceNode(info)

&lt;-ctx.Done()
}
```
  Implement the Target Node
  - In the runTargetNode function, add a new stream handler for the /hello/1.0.0 protocol. The stream handler of a protocol specifies what to do when a stream for that protocol is opened in the node. The SetStreamHandler method expects the ID of the protocol and the function to execute when a new message is received.
```
func runTargetNode() peer.AddrInfo {
	log.Printf("Creating target node...")
	targetNode := createNode()
	log.Printf("Target node created with ID '%s'", targetNode.ID().String())
// TO BE IMPLEMENTED: Set stream handler for the &quot;/hello/1.0.0&quot; protocol
targetNode.SetStreamHandler(&quot;/hello/1.0.0&quot;, func(s network.Stream) {
	log.Printf(&quot;/hello/1.0.0 stream created&quot;)
	err := readHelloProtocol(s)
	if err != nil {
		s.Reset()
	} else {
		s.Close()
	}
})

return *host.InfoFromHost(targetNode)
}
```
  In this tutorial, the /hello/1.0.0 protocol is used as an example. In a real world application, you can specify handlers for the custom protocols that your application uses.
  - Now, implement the readHelloProtocol function, which reads the content of the stream.
```
func readHelloProtocol(s network.Stream) error {
	// TO BE IMPLEMENTED: Read the stream and print its content
	buf := bufio.NewReader(s)
	message, err := buf.ReadString('\n')
	if err != nil {
		return err
	}
connection := s.Conn()

log.Printf(&quot;Message from '%s': %s&quot;, connection.RemotePeer().String(), message)
return nil
}
```
  In the function, a new reader is created and the content is read as a string until a line break (\n) is found. Then, the connection is used to find out which peer sent the message.
  Implement the Source Node
  - Let’s move to the runSourceNode function. The source node is created and connected to the target node by using the information provided as a parameter. Let’s open a stream for the /hello/1.0.0 protocol and send a message. The NewStream method expects a context, the ID of the peer to open the stream, and the ID of the protocol.
```
func runSourceNode(targetNodeInfo peer.AddrInfo) {
	log.Printf("Creating source node...")
	sourceNode := createNode()
	log.Printf("Source node created with ID '%s'", sourceNode.ID().String())
sourceNode.Connect(context.Background(), targetNodeInfo)

// TO BE IMPLEMENTED: Open stream and send message
stream, err := sourceNode.NewStream(context.Background(), targetNodeInfo.ID, &quot;/hello/1.0.0&quot;)
if err != nil {
	panic(err)
}

message := &quot;Hello from Launchpad!\n&quot;
log.Printf(&quot;Sending message...&quot;)
_, err = stream.Write([]byte(message))
if err != nil {
	panic(err)
}
}
```
  You can read the code like “use the target node’s connection to open a stream for the /hello/1.0.0 protocol”.
  Test the Application
  - Let’s test everything by running the Go application.
```
> go run  .
2022/09/08 12:08:02 Creating target node...
2022/09/08 12:08:02 Target node created with ID '12D3KooWF7r4SxND9Qf8uu9u3PtDduG5ET4NZf1Pw25BMqQEvXRP'
2022/09/08 12:08:02 Creating source node...
2022/09/08 12:08:02 Source node created with ID '12D3KooWMErSsjs4nPzTDjSvGcWipqyfJYxoSQJM57mTPWccEYQA'
2022/09/08 12:08:02 Sending message...
2022/09/08 12:08:02 /hello/1.0.0 stream created
2022/09/08 12:08:02 Message from '12D3KooWMErSsjs4nPzTDjSvGcWipqyfJYxoSQJM57mTPWccEYQA': Hello from Launchpad!
```
- Quiz & Resources
- Learning Objectives
- Introduction to IPLD
- Content Addressing & CIDs
- Graphs: Merkle DAGs
- IPLD & IPFS
- The Data Model & Codecs
- Background
  In this tutorial, you are going to use the IPFS command line API to experience IPLD pathing features which allow you navigate a directed acyclic graph (DAG). You will be navigating the DAG of the ipld.io website that is hosted on IPFS, thanks to a service called Fleek.co.
  Fleek.co can display a static website whose code is hosted on Github. So whenever there is a change to the ipld.io github repo, the changes get added to IPFS through Fleek and the new CID gets updated automatically in the DNS server. Therefore, as you go through the tutorial the CIDs may be different depending on when you go though it.
  Pre-Requisites
  - Install IPFS on the command line, if you have not done that yet, check out this tutorial
  - Download jq on the command line:
    Linux or WSL: sudo apt-get install jq
    Mac: brew install jq
  - Open the ipld.io website
  - Have the IPLD Gitub Repo on hand
  Video: IPLD Pathing
  Instructions
  Once you have installed kubo (formerly go-ipfs) and initiated and ipfs node, start up ipfs with the command ipfs daemon, then open a second terminal window to follow the instructions below.
  Install jq
  Jq is a tool to help the user view json text easier on the command line. You can go to stedolan.github.io/jq/download to find instructions for your operating system.
  Find the CID for ipld.io website
```
ipfs resolve /ipns/ipld.io
```
  You should see the following output in your CLI:
```
/ipfs/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg
```
  This command can help you find the root CID for any website that is hosted on IPFS through DNSLink. Fleek makes websites readily available on IPFS by auto-updating the DNSLink with the most recent CID.
  Inspect the Root
  Now that you have the root CID for the website ipld.io, you can inspect the DAG that makes up the data stored on IPFS.
```
ipfs dag get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg | jq
```
  You should see the following output in your CLI, which is a list of the CIDs below the root block:
```
{
  "Data": {
    "/": {
      "bytes": "CAE"
    }
  },
  "Links": [
    {
      "Hash": {
        "/": "QmY55i3pt9gJh4po9cdRJzZFafPAwF7V8etP9xQsnBZ8sD"
      },
      "Name": "FAQ",
      "Tsize": 15452
    },
    {
      "Hash": {
        "/": "QmVWdcKiK2mWpDXQj9DtHsrBqzA4hSLxov5juvaS4wrCZD"
      },
      "Name": "css",
      "Tsize": 85864
    },
…
…
{
"Hash": {
"/": "QmPNReXtnq8rDGJEbGGdYRxbiaZa7du1TZ782syYJodQM1"
},
"Name": "index.html",
"Tsize": 26494
},
{
"Hash": {
"/": "QmcVrQ8Bhk2S858P4sSedR9PNc8pq2zvLXpsTTPRnLZk86"
},
"Name": "x",
"Tsize": 685
}
]
}
```
  This has pulled in the next level of blocks connected to the root CID, and printed it in DAG-JSON format. We use the jq command to print the json in an easy-to-read layout.
  The original block is actually DAG-PB format which is used to construct UnixFS file data in IPFS. If you inspect the binary format of the data with ipfs block get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEq you will get illegible information, since it is raw DAG-PB bytes.
  DAG-PB has two top-level properties: Data and Links. The Data field has the actual bytes contained in a block. Each Link in DAG-PB has a name, a CID and a size. The Links contain the Hash or CID for the data in the next level below. The Hash property can also be inspected, leading you another level down (to the children) of content in that Merkle tree.
  You can see here the "Hash" or CID of the index.html page for the website.
  Load a Page With an IPFS Path
  When a client (i.e a website/a browser via an IPFS gateway) loads a root like in the following manner and doesn’t find a single page, it will look for an index.html link, which ipld.io has. In this case, the Links field is empty, but the Data field contains a lot of bytes so the client will load those in. The bytes are Base64 encoded.
```
ipfs dag get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/index.html | jq
```
  You should see the following output in your CLI:
```
{
    "Data": {
        "/": { "bytes": "CAIS0s4BPCFET0NUWVBF... "}
    },
    "Links": []
}
```
  If you want to navigate to any other block using IPFS pathing, just add the name of the link in the path, similar to what you would do in a linux filesystem.
  As an example, if you take a look at this directory in the IPLD website github repo:
  You can reach this same directory on the CLI by simply adding the filepath on the end of the root CID like so:
```
ipfs dag get QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/docs/codecs/known | jq
```
  This command traverses the IPLD structure & finds the CID of the /docs/codec/known directory on Github. The output that you see should match the file structure of the Github repo above.
  Load a Page with IPLD Pathing
  DAG-PB is the default data format for IPFS and has some special properties that makes IPFS pathing possible.
  When inspecting a CID, you can both look for named “Links” or specify which node you would like to traverse by identifying the link by the number.
  To do this, with the ipfs get command you add /ipld to the CID, and then specify which ordinal Link you want to retrieve by appending /Links/<number>/Hash.
  Try the command
```
ipfs dag get /ipld/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/Links/3/Hash | jq
```
  This will retrieve the information about the 4th link listed under the root CID of QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg, which has a name of "docs".
```
{
    "Data": {
        "/": { "bytes": "CAIS0s4BPCFET0NUWVBF..." }
    },
    "Links": []
}
```
  The Links is an array field, so we use the number 3 to identify which child block we want to access. In this case, it’s going to navigate into the Hash for "docs" (since it’s at an index of 3) that will load the block with that CID. You can see that you get the same block you would with IPFS pathing referring to the same block:
  Inspect the Data in a Block
  If you add /Hash/Data to the end of a CID you are accessing certain fields in the DAG-PB data structure, it will inspecting the raw bytes of a block. In this example, we are examining the block at an index of 7, which happens to be the index.html page:
```
ipfs dag get /ipld/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/Links/7/Hash/Data | jq
```
  You should see something like the following in your CLI:
```
{
    "/": {  "bytes": "CAIS0s4BPCFET0NUWVBF... "}
}
```
  Switching Codecs
  IPLD pathing, by default, allows you to inspect data in DAG-JSON format (kubo transforms DAG-PB into DAG-JSON in the CLI), and not the raw byte array. We are going to use a command to inspect the data of the index.html page in it’s original format.
  This command uses the pathing command ipfs dag get, with the flag --output-codec=raw to change the codec you use to view the data from DAG-JSON to the RAW codec and you are able to view the bytes as they are.
```
ipfs dag get --output-codec=raw /ipld/QmQ2ocFLq6d7ZiVEQfuEGEr4niJmdSscoyLkgTKRWmAEqg/Links/7/Hash/Data
��<!DOCTYPE html>
<html class="no-js" lang="en-US" prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb#">
<head>
…
<link rel="stylesheet" href="css/tachyons.min.css" />
<link href="https://fonts.googleapis.com/css?family=Open+Sans:400,600,700" rel="stylesheet"/>
<title>IPLD - The data model of the content-addressable web</title>
…
…
```
- IPLD Schemas
- Distributed Data Structures
- The CAR format
- IPLD Shallow and Deep Sessions
- Quiz & Resources

Data Discovery & Connections

Mutable Content

One of the most powerful things about IPFS is that any piece of data or content you store on the network cannot be modified without changing the Content Identifier (CID) for that data, since the CID is created (in part) by hashing the content. There are various ways to create mutable data on top of IPFS and in this lesson we will learn what just a few of them are.

Mutable Content | ResNetLabs on Tour – David Dias

See the full set of resources on the ResNetLab Tutorials page

Nodes, Peers, and the Swarm

A Peer is any connected node on IPFS that relays and/or stores information on the network. You can either search peers using the DHT and Kademlia, or be directly connected to a peer. The set of peers that you (as a peer) are connected to directly is called a Swarm.

IPFS Nodes are programs that run on a computer that can exchange data with other IPFS nodes. Bootstrap nodes are used when a new node initially enters the IPFS network.

The DHT

The public Distributed Hash Table is the record of content that is used, along with Kademlia, to discover content-addressed data in a peer-to-peer network. The DHT is the mechanism that allows a peer-to-peer network to work without the old client-server model that the web2 internet runs on.

IPFS and the DHT

The DHT is a distributed system for mapping keys to values. In IPFS, the DHT is used as the fundamental component of the content routing system. It maps what the user is looking for (a CID) to the peer that is actually storing the matching content. There are 3 types of key-value pairings that are mapped using the DHT:

Provider Records – These map a data identifier (i.e., a multihash) to a peer that has advertised that they have, and are willing, to provide you with that content. This is used by IPFS to find content, and IPNS to find pubsub peers
IPNS Records – These map an IPNS key (i.e., hash of a public key) to an IPNS record (i.e., a signed and versioned pointer to some path like /ipfs/bafyXYZ)
Peer Records – These map a peerID to a set of multiaddresses at which the peer may be reached. This is used by IPFS when we know of a peer with content, but do not know its address, and used for manual connections

Kademlia

Kademlia is a distributed hash table for decentralized peer-to-peer computer networks designed by Petar Maymounkov and David Mazières in 2002. It specifies the structure of the network and the exchange of information through node lookups.

Kademlia makes it easier and quicker to find peers with content by, essentially, comparing how similar two nodes’ content is and rank it by how similar or ‘close’ it is. Read the paper to learn about Kademlia more in-depth.

The InterPlanetary Name System (IPNS)

Since IPFS uses CIDs, if you were to share an IPFS address such as /ipfs/QmbezGequPwcsWo8UL4wDF6a8hYwM1hmbzYv2mnKkEWaUp with someone, you would need to give the person a new link every time you update the content, because every change would result in a new CID.

The InterPlanetary Name System (IPNS) solves this issue by creating a link that can be updated. Thus, IPNS bridges the gap to Web3 by providing functionality that Web2 users are already familiar with. For example, when you go to a users’ website, you expect to find that same website with the same link or URL in the future. If that website is updated, you will see those updates as well.

This “link” in IPNS is called a name, this name is the hash of a public key. The name is associated with a record containing information about the hash that it points to and is signed by the public key’s corresponding private key. This allows new records to be signed and published at any time. Using IPNS means that when someone searches for your website using your name, they will receive the most up-to-date content as expected in today’s internet. You can learn more about IPNS and how to use it here.

Pubsub + IPNS

Publish/Subscribe (PubSub) is a messaging protocol to quickly communicate with other peers. Whenever a peer Publishes a message, Subscribing peers will receive it almost instantly. This protocol is not specific to IPFS or IPNS, but to Libp2p; paired with IPNS it allows for quick delivery of records over the network. With PubSub enabled on IPNS, updates to a record can be shared virtually instantly with subscribers.

IPNS is a self-certifying mutable pointer. Meaning any name that gets published is signed by a private key and anyone else can verify that it was signed by that peer with just the name.

How it Works

Publishing - When you first publish an IPNS name, you create a brand new record containing the CID to point to,a timestamp, and a cryptographic signature of the record created with the private key to establish you are the owner and certify the name.
Searching - Kubo uses the DHT to find peers that will have the queried record. This method has gotten faster over the years, but is still limited by the speed of DHT resolution itself. Alternative transports can be used to improve resolution speed (see PubSub).
Validity - A record is valid for 24 hours by default, but you can change its validity to be longer. When someone has your record, but it is expired, they will have to go to DHT to find a new, valid version of your record.
Keys - A name is a hash of a public key in a key pair. By default, the first public key used by Kubo is the same as the one for identifying your peer (PeerID). You can generate new key pairs with Kubo and use them to create additional IPNS records.

To accomplish IPNS with PubSub, a persistence layer is added. Now when you ask for a name, you are subscribing to a PubSub topic based on that name, you create a connection with a peer that is following the same name, then they send you the latest version of the record.

Watch the Mutable Content video above to learn more about IPNS over PubSub

The Public DHT | LabWeek 2021

Edit this page on GitHub