Skip to content

Commit

Permalink
docs: open links in separate tab
Browse files Browse the repository at this point in the history
  • Loading branch information
MitchiLaser committed Aug 9, 2024
1 parent f3ddd90 commit c18fead
Showing 1 changed file with 10 additions and 10 deletions.
20 changes: 10 additions & 10 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,21 +34,21 @@ <h2>More details, please!</h2>
<h2>But I don't have two dozen computers at home to build my own cluster.</h2>
<p>Don't worry, you are not alone with this problem. The computer pool from the physics faculty at KIT consists of 34 PCs, and <em>PISA</em> was developed to distribute computational jobs among them. It uses SSH access to distribute jobs, and user home directories are synchronized through a network drive, eliminating the need to manually synchronize files within the network. With a cluster configuration, a file provided on this website for the computing pool, <em>PISA</em> can be used to combine any set of homogeneous computers with SSH access and operate as a batch system on this composition. The <em>PISA</em> application takes on the responsibility of distributing the workload among the available devices, restarting tasks if a remote machine fails to respond, and collecting the output of all jobs.</p>
<h2>Which rules do I have to follow when I want to use the computers?</h2>
<p>When you registered for an account, you agreed to the computing rules, which can be <a href="https://comp.physik.kit.edu/Account/Benutzerordnung.html" target="_blank">found here</a>. These rules allow to cluster the computers, provided it is for purposes related to your studies. Illegal activities, such as mining for cryptocurrencies or downloading copyrighted media, will be detected and the responsible user will be held accountable. If you have any concerns, you can always ask the administrator or the "poolraum-hiwi" for advice.</p>
<p>When you registered for an account, you agreed to the computing rules, which can be <a target="_blank" href="https://comp.physik.kit.edu/Account/Benutzerordnung.html" target="_blank">found here</a>. These rules allow to cluster the computers, provided it is for purposes related to your studies. Illegal activities, such as mining for cryptocurrencies or downloading copyrighted media, will be detected and the responsible user will be held accountable. If you have any concerns, you can always ask the administrator or the "poolraum-hiwi" for advice.</p>
<p>Furthermore, we ask you to be considerate of other users. The computers are also used for tutorials and practical courses, which cannot take place if they are overloaded. Therefore, we request that you limit resource allocation by running a limited number of jobs on each computer simultaneously, ensuring that no single person monopolizes the computers. If you require more computational power, you can perform your tasks with higher resource allocation overnight when no users are present, but make sure everything is completed by morning. If this is still insufficient, you might need to consider other platforms to run your code, or perhaps revisit the idea of building your own cluster.</p>
<h2>What kind of jobs are suitable for a batch system?</h2>
<p>A batch system is designed for running programs on other devices independently from each other. If your program needs to be executed only a single time, has a long runtime, and cannot be divided into individual tasks, then <em>PISA</em> cannot assist you. Design your Python script so that specific conditions, varying between each run, can be passed as command-line arguments, as <em>PISA</em> (and batch systems in general) do not support user input during runtime. For an easy way to parse command-line arguments within your script, check out the <a href="https://docs.python.org/3/library/argparse.html">Python argparse module</a>. Additionally, you will achieve higher benefits when your jobs are compute-bound, meaning they spend more time performing computational work rather than waiting for input/output operations on files, network interfaces, or memory access. <em>PISA</em> is designed to create a high-throughput system; for high-performance systems, there are other requirements.</p>
<p>A batch system is designed for running programs on other devices independently from each other. If your program needs to be executed only a single time, has a long runtime, and cannot be divided into individual tasks, then <em>PISA</em> cannot assist you. Design your Python script so that specific conditions, varying between each run, can be passed as command-line arguments, as <em>PISA</em> (and batch systems in general) do not support user input during runtime. For an easy way to parse command-line arguments within your script, check out the <a target="_blank" href="https://docs.python.org/3/library/argparse.html">Python argparse module</a>. Additionally, you will achieve higher benefits when your jobs are compute-bound, meaning they spend more time performing computational work rather than waiting for input/output operations on files, network interfaces, or memory access. <em>PISA</em> is designed to create a high-throughput system; for high-performance systems, there are other requirements.</p>
</article>

<article id="prerequsites">
<h1>Prerequisites</h1>
<p>Before you can use <em>PISA</em>, you need to have a few things set up.</p>

<h2>SSH</h2>
<p>To use <em>PISA</em>, you first need to configure SSH access between all devices to authenticate with a key file. Without this configuration, each submitted job would require a human to type in the login password to establish the connection. With a trusted SSH key configured, the authentication process replaces the password prompt with the key file. For more information about establishing an SSH connection, please refer to the <a href="https://spice-space.de/inhalt/physik-pool/">Instructions for using SSH for the physics computer pool</a>. Once you understand how to establish an SSH connection, you need to set up <a href="https://spice-space.de/inhalt/physik-pool/#key-login">passwordless login to remote devices via the SSH key</a>. If you are unsure how to proceed you can just open a terminal and call <code>enable_ssh_key_pool.sh</code> from anywhere, it should be installed on all computers in the pool. More details about this script can be read in the Instructions for using SSH for the physics computer pool (link above).</p>
<p>To use <em>PISA</em>, you first need to configure SSH access between all devices to authenticate with a key file. Without this configuration, each submitted job would require a human to type in the login password to establish the connection. With a trusted SSH key configured, the authentication process replaces the password prompt with the key file. For more information about establishing an SSH connection, please refer to the <a target="_blank" href="https://spice-space.de/inhalt/physik-pool/">Instructions for using SSH for the physics computer pool</a>. Once you understand how to establish an SSH connection, you need to set up <a target="_blank" href="https://spice-space.de/inhalt/physik-pool/#key-login">passwordless login to remote devices via the SSH key</a>. If you are unsure how to proceed you can just open a terminal and call <code>enable_ssh_key_pool.sh</code> from anywhere, it should be installed on all computers in the pool. More details about this script can be read in the Instructions for using SSH for the physics computer pool (link above).</p>

<h2>Virtual Environment</h2>
<p>Your Python environment needs to be consistent across all devices where you want to run your programs. This is important to ensure that all Python packages required by your application are available. <em>PISA</em> relies on <a href="https://docs.python.org/3/library/venv.html">Python virtual environments</a> (venv) to provide a uniform operational basis for all tasks executed remotely. If you are unfamiliar with virtual environments, it is highly recommended to become familiar with them. In this setup, your virtual environment is tied to the source code of your project. Assuming your Python source files are located in a directory, open a shell in that directory and run:</p>
<p>Your Python environment needs to be consistent across all devices where you want to run your programs. This is important to ensure that all Python packages required by your application are available. <em>PISA</em> relies on <a target="_blank" href="https://docs.python.org/3/library/venv.html">Python virtual environments</a> (venv) to provide a uniform operational basis for all tasks executed remotely. If you are unfamiliar with virtual environments, it is highly recommended to become familiar with them. In this setup, your virtual environment is tied to the source code of your project. Assuming your Python source files are located in a directory, open a shell in that directory and run:</p>
<p class="codeblock shellcode"><code>python -m venv venv/</code></p>
<p>This command creates a virtual environment. You will notice a new directory named "venv" (the last argument is the directory). To activate the virtual environment, run:</p>
<p class="codeblock shellcode"><code>source ./venv/bin/activate</code></p>
Expand All @@ -64,7 +64,7 @@ <h2>Step 1: Preparation</h2>
<p>You can install <em>PISA</em> globally or within your project-specific venv; it just needs to be callable.</p>

<h2>Step 2: Cluster configuration</h2>
<p><em>PISA</em> needs to know which computers it can connect to. The set of available machines is specified in a cluster configuration file when <em>PISA</em> is executed. There is a <a href="https://github.com/MitchiLaser/pisa/blob/main/config/fphct_cluster.json">predefined configuration</a> file for the computer pool at the physics faculty at KIT that can be downloaded from the command line:</p>
<p><em>PISA</em> needs to know which computers it can connect to. The set of available machines is specified in a cluster configuration file when <em>PISA</em> is executed. There is a <a target="_blank" href="https://github.com/MitchiLaser/pisa/blob/main/config/fphct_cluster.json">predefined configuration</a> file for the computer pool at the physics faculty at KIT that can be downloaded from the command line:</p>
<p class="codeblock shellcode"><code>wget https://raw.githubusercontent.com/MitchiLaser/pisa/main/config/fphct_cluster.json</code></p>

<h2>Step 3: Job description file</h2>
Expand All @@ -76,7 +76,7 @@ <h2>Step 3: Job description file</h2>
<li>Any command line arguments that are the same for each run.</li>
<li>Any command line arguments that vary for each run and the values that should be passed to the program.</li>
</ul>
<p>The output of all jobs is stored in files, and the assignment of each run to its corresponding command line arguments is stored in an assignment file, generated while <em>PISA</em> is running. To provide <em>PISA</em> with the necessary information, a job description file is passed to <em>PISA</em> when it is executed. An <a href="https://github.com/MitchiLaser/pisa/blob/main/config/example_task.toml">example job description file</a> for a simple <a href="https://github.com/MitchiLaser/pisa/blob/main/config/fib.py">example program</a> (a Fibonacci number calculator with poor runtime) is provided. This file needs to be adjusted for each batch of jobs that <em>PISA</em> should process.</p>
<p>The output of all jobs is stored in files, and the assignment of each run to its corresponding command line arguments is stored in an assignment file, generated while <em>PISA</em> is running. To provide <em>PISA</em> with the necessary information, a job description file is passed to <em>PISA</em> when it is executed. An <a target="_blank" href="https://github.com/MitchiLaser/pisa/blob/main/config/example_task.toml">example job description file</a> for a simple <a target="_blank" href="https://github.com/MitchiLaser/pisa/blob/main/config/fib.py">example program</a> (a Fibonacci number calculator with poor runtime) is provided. This file needs to be adjusted for each batch of jobs that <em>PISA</em> should process.</p>
<p>To simplify the structure of the job description file, all file locations are specified relative to a working directory. <em>PISA</em> can handle any number of variable arguments for each run.</p>

<h2>Step 4: Submit your jobs</h2>
Expand All @@ -88,18 +88,18 @@ <h2>Step 4: Submit your jobs</h2>

<article id="example">
<h1>Example program</h1>
<p>The <a href="https://github.com/MitchiLaser/pisa/tree/main/config">GitHub repository</a> contains a directory with an example program, the corresponding task description file, and the cluster configuration file for the fphct computing pool. Check out these files to familiarize yourself with the use of <em>PISA</em>. To run the example, a virtual environment needs to be created, and the file locations should match the ones in the task description file.</p>
<p>The <a target="_blank" href="https://github.com/MitchiLaser/pisa/tree/main/config">GitHub repository</a> contains a directory with an example program, the corresponding task description file, and the cluster configuration file for the fphct computing pool. Check out these files to familiarize yourself with the use of <em>PISA</em>. To run the example, a virtual environment needs to be created, and the file locations should match the ones in the task description file.</p>
</article>

<article id="development">
<h1><em>PISA</em> development</h1>
<p><em>PISA</em> is an open-source project. If you encounter any issues with <em>PISA</em> or have feature requests, you can <a href="https://github.com/MitchiLaser/pisa/issues">submit an issue</a> or implement the solution yourself and make a <a href="https://github.com/MitchiLaser/pisa/pulls">pull request</a>.</p>
<p><em>PISA</em> is an open-source project. If you encounter any issues with <em>PISA</em> or have feature requests, you can <a target="_blank" href="https://github.com/MitchiLaser/pisa/issues">submit an issue</a> or implement the solution yourself and make a <a target="_blank" href="https://github.com/MitchiLaser/pisa/pulls">pull request</a>.</p>
</article>

</main>
<footer>
<p><a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons Lizenzvertrag" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a>&nbsp;<a href="https://www.spice-space.de/navigation/ueber-michi/">Michael Hohenstein</a>, 2024</p>
<p><a href="https://docs.github.com/en/site-policy/privacy-policies/github-general-privacy-statement">Privacy</a></p>
<p><a target="_blank" rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons Lizenzvertrag" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a>&nbsp;<a target="_blank" href="https://www.spice-space.de/navigation/ueber-michi/">Michael Hohenstein</a>, 2024</p>
<p><a target="_blank" href="https://docs.github.com/en/site-policy/privacy-policies/github-general-privacy-statement">Privacy</a></p>
</footer>
</body>
</html>

0 comments on commit c18fead

Please sign in to comment.