Step 5: Verify installation. If everything worked fine you will be able to open a spark-shell running the following command. $ spark-shell 20/04/21 12:32:33 WARN Utils: Your hostname, mac.local resolves to a loopback address: 127.0.0.1; using 192.168.1.134 instead (on interface en1) 20/04/21 12:32:33 WARN Utils: Set SPARKLOCALIP if you need. MacOS, Brew, Supervisor, Laravel & Horizon May 29, 2019 - 5:59 pm Ark: Survival Evolved – Server Settings for 800 Levels January 11, 2017 - 5:29 am Categories.
The command line interface is a non-graphical way to interact with your computer. Instead of clicking buttons with your mouse, you’ll type commands as text and receive text-based feedback. The command line, also known as a shell, lets you automate many tasks you do on your computer daily, and is an essential tool for software developers.
While the command line interface on macOS has a lot of the functionality you’d find in Linux and other Unix systems, it does not ship with a package manager. A package manager is a collection of software tools that work to automate software installations, configurations, and upgrades. Package managers keep the software they install in a central location and can maintain all software packages on the system in formats that are commonly used.
Homebrew is a package manager for macOS which lets you install free and open-source software using your terminal. You’ll use Homebrew to install developer tools like Python, Ruby, Node.js, and more.
In this tutorial you’ll install and use Homebrew on your Mac. You’ll install system tools and desktop applications from the command line interface.
You will need a macOS computer running Catalina or higher with administrative access and an internet connection. While older versions of macOS may work, they are not officially supported.
To access the command line interface on your Mac, you’ll use the Terminal application provided by macOS. Like any other application, you can find it by going into Finder, navigating to the
Applications folder, and then into the
Utilities folder. From here, double-click the Terminal application to open it up. Alternatively, you can use Spotlight by holding down the
COMMAND key and pressing
SPACE to find Terminal by typing it out in the box that appears.
To get more comfortable using the command line, take a look at An Introduction to the Linux Terminal. The command line interface on macOS is very similar, and the concepts in that tutorial are directly applicable.
Now that you have the Terminal running, let’s install some additional tools that Homebrew needs.
Xcode is an integrated development environment (IDE) that is comprised of software development tools for macOS. You won’t need Xcode to use Homebrew, but some of the software and components you’ll want to install will rely on Xcode’s Command Line Tools package.
Execute the following command in the Terminal to download and install these components:
You’ll be prompted to start the installation, and then prompted again to accept a software license. Then the tools will download and install automatically.
You can now install Homebrew.
To install Homebrew, you’ll download an installation script and then execute the script.
First, download the script to your local machine by typing the following command in your Terminal window:
The command uses
curl to download the Homebrew installation script from Homebrew’s Git repository on GitHub.
Let’s walk through the flags that are associated with the
--failflag tells the Terminal window to give no HTML document output on server errors.
curlso that it does not show the progress meter, and combined with the
--show-errorflag it will ensure that
curlshows an error message if it fails.
--locationflag will tell
curlto handle redirects. If the server reports that the requested page has moved to a different location, it’ll automatically execute the request again using the new location.
-oswitch specifies a local filename for the file. Rather than displaying the contents to the screen, the
-oswitch saves the contents into the file you specify.
Before running a script you’ve download from the Internet, you should review its contents so you know what the script will do. Use the
less command to review the installation script so you understand what it will do'
Once you’re comfortable with the contents of the script, execute the script with the
The installation script will explain what it will do and will prompt you to confirm that you want to do it. This lets you know exactly what Homebrew is going to do to your system before you let it proceed. It also ensures you have the prerequisites in place before it continues.
You’ll be prompted to enter your password during the process. However, when you type your password, your keystrokes will not display in the Terminal window. This is a security measure and is something you’ll see often when prompted for passwords on the command line. Even though you don’t see them, your keystrokes are being recorded by the system, so press the
RETURN key once you’ve entered your password.
Press the letter
y for “yes” whenever you are prompted to confirm the installation.
Once the installation process is complete, you will want to put the directory Homebrew uses to store its executables at the front of the
PATH environment variable. This ensures that Homebrew installations will be called over the tools that macOS includes.
The file you’ll modify depends on which shell you’re using. ZSH is the default shell on macOS Mojave and higher. The Bash shell is a popular shell that older versions of macOS used as the default, and if you’ve upgraded your OS, you may still be using Bash.
Execute the following command to determine your shell:
You’ll see either
If you’re using ZSH, you’ll open the file
~/.zshrc in your editor:
If you’re using the Bash shell, you’ll use the file
Once the file opens up in the Terminal window, add the following lines to the end of the file:
The first line is a comment that will help you remember what this does if you open this file in the future.
To save your changes, hold down the
CTRL key and the letter
O, and when prompted, press the
RETURN key. Then exit the editor by holding the
CTRL key and pressing
X. This will return you to your Terminal prompt.
To activate these changes, close and reopen your Terminal app. Alternatively, use the
source command to load the file you modified.
If you modified
.zshrc, execute this command:
If you modified
.bash_profile, execute this command:
Once you have done this, the changes you have made to the
PATH environment variable will take effect. They’ll be set correctly when you log in again in the future, as the configuration file for your shell is executed automatically when you open the Terminal app.
Now let’s verify that Homebrew is set up correctly. Execute this command:
If no updates are required at this time, you’ll see this in your Terminal:
Otherwise, you may get a warning to run another command such as
brew update to ensure that your installation of Homebrew is up to date. Follow any on-screen instructions to fix your environment before moving on.
Now that Homebrew is installed, use it to download a package. The
tree command lets you see a graphical directory tree and is available via Homebrew.
tree with the
brew install command:
Homebrew will update its list of packages and then download and install the
Homebrew installs files to
/usr/local by default, so they won’t interfere with future macOS updates. Verify that
tree is installed by displaying the command’s location with the
The output shows that
tree is located in
tree command to see the version:
The version prints to the screen, indicating it’s installed:
Occasionally, you’ll want to upgrade an existing package. Use the
brew upgrade command, followed by the package name:
You can run
brew upgrade with no additional arguments to upgrade all programs and packages Homebrew manages.
When you install a new version, Homebrew keeps the older version around. After a while, you might want to reclaim disk space by removing these older copies. Run
brew cleanup to remove all old versions of your Homebrew-managed software.
To remove a package you’re no longer using, use
brew uninstall. To uninstall the
tree command, execute this command:
The output shows that the package was removed:
You can use Homebrew to install desktop applications too.
You’re not restricted to using Homebrew for command-line tools. Homebrew Cask lets you install desktop applications. This feature is included with Homebrew, so there’s nothing additional to install.
Test it out by using Homebrew to install Visual Studio Code. Execute the following command in your terminal:
The application will install:
You’ll find the application in your
Applications folder, just as if you’d installed it manually.
To remove it, use
Homebrew will remove the installed software:
It performs a backup first in case the removal fails, but once the program is fully uninstalled, the backup is removed as well.
If you no longer need Homebrew, you can use its uninstall script.
Download the uninstall script with
As always, review the contents of the script with the
less command to verify the script’s contents:
Once you’ve verified the script, execute the script with the
--help flag to see the various options you can use:
The options display on the screen:
-d flag to see what the script will do:
The script will list everything it will delete:
When you’re ready to remove everything, execute the script without any flags:
This removes Homebrew and any programs you’ve installed with it.
In this tutorial you installed and used Homebrew on your Mac. You can now use Homebrew to install command line tools, programming languages, and other utilities you’ll need for software development.
Homebrew has many packages you can install. Visit the official list to search for your favorite programs.
This short guide will assume that you already have already
java installed on your macOS. If not, run the following commands on your terminal.
Once you are sure that everything is correctly installed on your machine, you have to follow these steps to install Apache Spark
Add the following environment variables to your
For some reason, some installations are not give execution permission to binaries. Review it and give them if necessary.
If everything worked fine you will be able to open a
spark-shell running the following command
This should open a shell as follows
We are done! You have successfully installed Apache Spark on macOS
You can also find me on Twitter if you’d like to read similar technical tricks!