Donate VRC/VRM for the Wiki to VBzaNDExHyFpnNvYc5QH5e4ipBZqxxPnKJ . Thank you, Joe.

Difference between revisions of "Linux System Administration"

From VeriCoin & Verium Wiki
Jump to: navigation, search
(GUI or Command Line?: +tip:theFuck)
m (Package Management: make pretty the command references)
Line 74: Line 74:
 
Each Linux distribution has a different scheme for managing installation and removal of software, collectively referred to as 'packages.' Most distributions use apt or rpm. There are others, some of which are gaining ground cross-distribution.
 
Each Linux distribution has a different scheme for managing installation and removal of software, collectively referred to as 'packages.' Most distributions use apt or rpm. There are others, some of which are gaining ground cross-distribution.
  
If you have a Debian-based distribution, your package manager is apt. Your command-line packaging commands will mostly all be something similar to <code>apt [some command] [some package name or list of names] [optional arguments]</code>.
+
If you have a Debian-based distribution, your package manager is apt. Your command-line packaging commands will mostly all be something similar to
 +
  apt [some command] [some package name or list of names] [optional arguments]
  
If you have a Red Hat -based distribution (RHEL, CentOS, Fedora), your package manager is rpm. Confusingly, your command-line packaging commands will mostly all be similar to <code>yum [some command] [optional arguments] [some package name or list of names]</code>.
+
If you have a Red Hat -based distribution (RHEL, CentOS, Fedora), your package manager is rpm. Confusingly, your command-line packaging commands will mostly all be similar to
 +
  yum [some command] [optional arguments] [some package name or list of names]
  
 
These are the two most common package management systems in the Linux world. There are others. Arch has pacman. Slackware use pkgtool. Gentoo has emerge. The BSD and Solaris (UNIX, not Linux) systems have pkg and ports. Be sure to check your distro's documentation to learn how to use the package manager.
 
These are the two most common package management systems in the Linux world. There are others. Arch has pacman. Slackware use pkgtool. Gentoo has emerge. The BSD and Solaris (UNIX, not Linux) systems have pkg and ports. Be sure to check your distro's documentation to learn how to use the package manager.

Revision as of 11:40, 26 January 2018

Working with Linux

Congratulation! You're a nerd! You are a deviant Linux user and probably are on an FBI or CIA watch list. Not that it matters, because you're also into crypto-currencies which gets you onto the terrorist watch lists, anyway, so why worry?

It turns out all of this stuff is important to maximizing your system for mining performance or monitoring your miners or just making your administration easier. However. None of it is really within the purpose and mission of the wiki. This section mostly contains some basic intro/overview text and links to third-party sources for further information. As much as possible, the section authors have tried to link to official documentation or man pages. These, especially man pages, are often hard to decipher and put into effective action. There may or may not eventually be configuration how-to pages added to the wiki, but this page is really not the most appropriate for OS-level tweaks; just wallet and mining software tweaks.

That said, there are people who are very knowledgeable, helpful, and active in the Slack channel. Your distribution (or SBC vendor's) user-forums are also excellent sources for help. Weirdly, remember to check YouTube; it has plenty of good tutorials.

Did I mention the Arch Linux wiki? Go and explore. I doubt there is, anywhere, a better source of technical how-to information for the Linux enthusiast. Just translate the package installation commands into whatever distribution you use.

GUI or Command Line?

Linux has lots of options for user interfaces. You probably want a pretty, graphical interface for your daily-use computer, spending-wallet software, and maybe also the machine you monitor your mining rigs with and/or your cluster's edge server.

But, if you run a GUI on mining rigs, that GUI is using resources your mining software probably wants for itself. Mining rigs should be kept lean and mean, like a top-fuel dragster or Formula One race car. Unless you're just hobby mining during the day while you're away at work or at night while you sleep. A dedicated mining rig wants a command line shell for a user interface.

Everyone's use-case for their hardware is different. Everyone makes different trade-offs in performance vs. maintainability. Every machine is unique; even the individual slave nodes in your cluster, if you have one.

-- GUI Options

GNOME and KDE are the big kids on the block. They also require the most memory, CPU, and graphical processing power. Lighter weight options like Budgie and LXDE and XFCE also are common desktop environments. If you don't already have a favorite, Wikipedia's list of DEs can help you.

Personally, I recommend you stay away from any of these for a server. No need to waste the resources. If you need to use GUI applications, there are plenty of lighter weight options than full-blown desktop environments. Get out your favorite search engine and look up "tiling window manager". You'll find such wonderful names as 'ratpoison' and 'awesome.' Go ahead. Try them out in a VM to find one you like.

emacs fans will no doubt poo-poo the lightweight GUI thing, and just use emacs for everything. More. Power. To you. Just remember: evil-mode is your friend. :)

-- Command Shells

bash. fish. zsh. ksh. What? That list reads like action pop-outs in a comic book. These are names of command-line shell environments. Each one has a different set of built-in utility functions. Each has different scripting functionality and configuration options. Yes, you read that right: configuration options. A shell environment can be a very personal space.

Most Linux distributions currently ship with and default to bash. It's a solid general purpose shell environment. That doesn't make it the best shell environment. Most Linux users never use anything else.

If you find yourself doing lots of work in the command shell, or writing lots of shell scripts, do yourself a favor and familiarize yourself with the features of other shells. Future-you will thank current/past-you for doing so.

All that being said, this is where you want your mining rigs. A basic shell environment is the lowest resource-use human interface available. If you're mining, there's a good chance you'll spend a bunch of time here. If you have a cluster of miners or want to make a cluster of your current swarm of individuals, you're going spend time here.

Settle in. Get comfortable.

Now you've done that, look into various multi-shell management programs. They also are very light on system resources. They can save your bacon when trying to several things at once across a ssh session from halfway around the world, or from across the room.

GNU's screen utility (quick reference) comes preinstalled on most distributions. It's a solid performer, but it has limitations. tmux (quick reference) is a more modern - well, less OLD - option. It is only sometimes preinstalled. There are other options. Search for "terminal multiplexer".

Just a quick tip: Install Fuck. Really. It will save you endless typing if you are prone

 $ ot typso
 unknown command
 $ fuck
 $ to typos

on the command line.

-- Text Editors

Why, you might ask, is this even here? You might think text editors are boring. You'd be wrong. Did you know, there are people who happily shell out $300, or more, for their favorite MacOS, Windows, and even -gasp!- Linux text editor???

As a Linux user, you're going to be using a text editor a lot. Way more than the average Windows or Mac person. Most configuration is done, or can be done, by editing text files.

Text. Plaintext. Markdown. ReSructuredText. Asciidoc. HTML. XML. json. Docbook. LaTeX. epub. All of these are plain-old, boring text; and that's a very short list of text formats. There's a *very long* list of different textual-format file types. Text is king, and for good reason: text is very easy, bordering on trivial, to search and parse programmatically. That's why it's used for configuration files. And config files are part of why you'll spend so much time in a text editor.

There are some really good graphical text editors; Atom, Gedit and Sublime Text are three of the most popular. They're pretty. They're featureful. They're configurable - some more than others. But they aren't available in the shell, over an ssh connection. For that, you want something different.

Your Linux distribution probably ships with nano and vi. Of the two, vi is the more featureful, but it also has a steep (REALLY steep) learning curve. vi is absolutely, always, available on EVERY Linux, BSD or other UNIX -based computer unless specifically and intentionally removed (sometimes with malice) by the sysadmin. That's not the case with nano. You should get familiar with the bare basics of using vi.

vim is an upgrade to vi. It has the same commands. It's extremely configurable. It can be scripted (has its own scripting language) and millions of users swear by it. It might be, but probably isn't, preinstalled on your distribution.

emacs is almost never preinstalled on a system; which is a shame. It is, arguably, the most capable program available on Linux or BSD. It's a supremely capable, configurable, scriptable text editor. It has a built-in web browser, email client, news reader, note taker, and more. It also has a very steep learning curve.

The vim/emacs war (which is better?) continues to rage. Ultimately, emacs is more capable and easier to script - or, at least, easier to debug those scripts. It has a mode, called EviL (not installed by default; ever), which implements vi/m-like commands.

Learn the basics of vi/m. Really. Just do it.

System Administration

Just the basics, here.

Package Management

Each Linux distribution has a different scheme for managing installation and removal of software, collectively referred to as 'packages.' Most distributions use apt or rpm. There are others, some of which are gaining ground cross-distribution.

If you have a Debian-based distribution, your package manager is apt. Your command-line packaging commands will mostly all be something similar to

 apt [some command] [some package name or list of names] [optional arguments]

If you have a Red Hat -based distribution (RHEL, CentOS, Fedora), your package manager is rpm. Confusingly, your command-line packaging commands will mostly all be similar to

 yum [some command] [optional arguments] [some package name or list of names]

These are the two most common package management systems in the Linux world. There are others. Arch has pacman. Slackware use pkgtool. Gentoo has emerge. The BSD and Solaris (UNIX, not Linux) systems have pkg and ports. Be sure to check your distro's documentation to learn how to use the package manager.

A relatively new, cross-distribution packaging system has gained some popularity: snap. Snap comes preinstalled on Ubuntu 16.04 and newer, but may also be preinstalled on other distros. If it's not preinstalled, and you want to use it, it's available in most package repositories.

Speaking of repositories... Your package manager gets all its package data, and often the packages themselves, from a centralized package repository somewhere out in internet-land. Repository management is also something you'll undoubtedly be required to do some of. Again, check your distro's documentation on how to go about doing this.

If you use a graphical package manager (GNOME Software, for example), much of this is hidden from you. If you're compiling mining software or wallets, you'll need to know the basics of package management in the terminal (command line.)

Basic Networking

  • nmTUI
  • avahi
  • /etc/hosts & /etc/hostname
  • etc. (advanced networking in Clustering)

Security & Hardening

  • PAM
  • firewalls (ufw and iptables are most common?)
  • intrusion detection and resistance (fail2ban ?)
  • SSH basics (save advance usage for Clustering section)
  • user/group permissions
  • jail and container intro (more detail in own section)
  • etc.

Backup, Fail-Over, & Disaster Recovery

  • backups
  • disk & system images

Remote Administration

  • basic ssh
  • scp
  • VNC and RDP
  • webmin and other resources

Advanced Topics

Clustering

  • webmin Cluster modules
  • ansible, chef, etc.
  • SSH, pSSH, SSHfs, gpg-key passwordless logins
  • NFS
  • PXE-booting (not supported most SBCs, but definitely supported most servers)
  • More advanced networking: DNS, DHCP, NTP, NAT (?)
  • see Containers and Virtual Machines

Containers and Virtual Machines

  • Pros, Cons of containers vs. VMs vs. bare metal
  • Available container technologies, pros/cons of each
    • lxc (personal favorite)
    • Docker
    • BSD jails
    • snapd containerized user-space applications
    • Management and provisioning tools (virt-engine, swarm, kubernetes, etc)
  • Available VMs, pros/cons of each
    • KVM
    • VMware/Vagrant
    • VirtualBox
    • Management and provisioning tools (virt-engine, swarm, kubernetes, etc)

Tuning and Pruning

Getting the leanest, meanest OS you can still make use of.

  • proprietary drivers


Hardware Control and Tuning

  • BIOS tweaks, upgrades
  • fan speed control
  • CPU throttling, overclocking and heat!
  • RAM throttling, overclocking and heat!
  • swap and swappiness
  • RAM-disks
  • HDD tuning - platters and SSDs
  • DRM and another GPU-related intro article (link to external resource for more depth)
  • PSU sizing for maximum reliability and longevity
  • battery backup and graceful shutdown (also sizing the battery)