|
|
[{"id":0,"href":"/cs341/en/chap1/Authors/","title":"Authors","section":"Introduction","content":" Authors # Bhuvan Venkatesh \u0026lt;bhuvan.venkatesh21@gmail.com\u0026gt; Lawrence Angrave \u0026lt;angrave@illinois.edu\u0026gt; joebenassi \u0026lt;joebenassi@gmail.com\u0026gt; jakebailey \u0026lt;zikaeroh@gmail.com\u0026gt; Ebrahim Byagowi \u0026lt;ebrahim@gnu.org\u0026gt; Alex Kizer \u0026lt;the.alex.kizer@gmail.com\u0026gt; dimyr7 \u0026lt;dimyr7.puma@gmail.com\u0026gt; Ed K \u0026lt;ed.karrels@gmail.com\u0026gt; ace-n \u0026lt;nassri2@illinois.edu\u0026gt; josephmilla \u0026lt;jjtmilla@gmail.com\u0026gt; Thomas Liu \u0026lt;thomasliu02@gmail.com\u0026gt; Johnny Chang \u0026lt;johnny@johnnychang.com\u0026gt; goldcase \u0026lt;johnny@johnnychang.com\u0026gt; vassimladenov \u0026lt;vassi1995@icloud.com\u0026gt; SurtaiHan \u0026lt;surtai.han@gmail.com\u0026gt; Brandon Chong \u0026lt;bchong95@users.noreply.github.com\u0026gt; Ben Kurtovic \u0026lt;ben.kurtovic@gmail.com\u0026gt; dprorok2 \u0026lt;dprorok2@illinois.edu\u0026gt; anchal-agrawal \u0026lt;aagrawa4@illinois.edu\u0026gt; Lawrence Angrave \u0026lt;angrave@illinois.eduuutoomanyu\u0026gt; daeyun \u0026lt;daeyunshin@gmail.com\u0026gt; bchong95 \u0026lt;bschong2@illinois.edu\u0026gt; rushingseas8 \u0026lt;georgealeks@hotmail.com\u0026gt; lukspdev \u0026lt;lllluuukke@gmail.com\u0026gt; hilalh \u0026lt;habashi2@illinois.edu\u0026gt; dimyr7 \u0026lt;dimyr7@hotmail.com\u0026gt; Azrakal \u0026lt;genxswordsman@hotmail.com\u0026gt; G. Carl Evans \u0026lt;gcevans@gmail.com\u0026gt; Cornel Punga \u0026lt;cornel.punga@gmail.com\u0026gt; vikasagartha \u0026lt;vikasagartha@gmail.com\u0026gt; dyarbrough93 \u0026lt;dyarbrough93@yahoo.com\u0026gt; berwin7996 \u0026lt;berwin7996@gmail.com\u0026gt; Sudarshan Govindaprasad \u0026lt;SudarshanGp@users.noreply.github.com\u0026gt; NMyren \u0026lt;ntmyren@gmail.com\u0026gt; Ankit Gohel \u0026lt;ankitgohel1996@gmail.com\u0026gt; vha-weh-shh \u0026lt;bhaweshchhetri1@gmail.com\u0026gt; sasankc \u0026lt;sasank.chundi@gmail.com\u0026gt; rishabhjain2795 \u0026lt;rishabhjain2795@gmail.com\u0026gt; nickgarfield \u0026lt;nickgarfield@icloud.com\u0026gt; by700git \u0026lt;aaabox@yeah.net\u0026gt; bw-vbnm \u0026lt;bwang19@illinois.edu\u0026gt; Navneeth Jayendran \u0026lt;jayndrn2@illinois.edu\u0026gt; Joe Benassi \u0026lt;joebenassi@gmail.com\u0026gt; Harpreet Singh \u0026lt;hshssingh4@gmail.com\u0026gt; FenixFeather \u0026lt;thomasliu02@gmail.com\u0026gt; EntangledLight \u0026lt;bdelapor@illinois.edu\u0026gt; Bliss Chapman \u0026lt;bliss.chapman@gmail.com\u0026gt; zikaeroh \u0026lt;zikaeroh@gmail.com\u0026gt; time bandit \u0026lt;radicalrafi@gmail.com\u0026gt; paultgibbons \u0026lt;paultgibbons@gmail.com\u0026gt; kevinwang \u0026lt;kevin@kevinwang.com\u0026gt; cPolaris \u0026lt;cPolaris@users.noreply.github.com\u0026gt; Zecheng (張澤成) \u0026lt;zzhan147@illinois.edu\u0026gt; Wieschie \u0026lt;supernova190@gmail.com\u0026gt; WeiL \u0026lt;z920631580@gmail.com\u0026gt; Graham Dyer \u0026lt;gdyer2@illinois.edu\u0026gt; Arun Prakash Jana \u0026lt;engineerarun@gmail.com\u0026gt; Ankit Goel \u0026lt;ankitgoel616@gmail.com\u0026gt; Allen Kleiner \u0026lt;akleiner24@gmail.com\u0026gt; Abhishek Deep Nigam \u0026lt;adn5327@users.noreply.github.com\u0026gt; zmmille2 \u0026lt;zmmille2@gmail.com\u0026gt; sidewallme \u0026lt;sidewallme@gmail.com\u0026gt; raych05 \u0026lt;raymondcheng05@gmail.com\u0026gt; mmahes \u0026lt;malinixmahes@gmail.com\u0026gt; mass \u0026lt;amass1212@gmail.com\u0026gt; kovaka \u0026lt;jakelagrou@gmail.com\u0026gt; gmag23 \u0026lt;gmag23@gmail.com\u0026gt; ejian2 \u0026lt;ejian2@illinois.edu\u0026gt; cerutii \u0026lt;marc.ceruti@gmail.com\u0026gt; briantruong777 \u0026lt;briantruong777@gmail.com\u0026gt; adevar \u0026lt;adevar2@illinois.edu\u0026gt; Yuxuan Zou (Sean) \u0026lt;yzouac@connect.ust.hk\u0026gt; Xikun Zhang \u0026lt;xikunz2@illinois.edu\u0026gt; Vishal Disawar \u0026lt;disawar2@illinois.edu\u0026gt; Taemin Shin \u0026lt;cprayer@naver.com\u0026gt; Sujay Patwardhan \u0026lt;sujay.patwardhan@gmail.com\u0026gt; SufeiZ \u0026lt;sufeizhang92@gmail.com\u0026gt; Sufei Zhang \u0026lt;sufeizhang92@gmail.com\u0026gt; Steven Shang \u0026lt;sstevenshang@users.noreply.github.com\u0026gt; Steve Zhu \u0026lt;st.zhu1@gmail.com\u0026gt; Sibo Wang \u0026lt;sibowsb@gmail.com\u0026gt; Shane Ryan \u0026lt;shane1027@users.noreply.github.com\u0026gt; Scott Bigelow \u0026lt;epheph@gmail.com\u0026gt; Riyad Shauk \u0026lt;riyadshauk@users.noreply.github.com\u0026gt; Nathan Somers \u0026lt;nsomers2@illinois.edu\u0026gt; LieutenantChips \u0026lt;vkaraku2@illinois.edu\u0026gt; Jacob K LaGrou \u0026lt;jakelagrou@gmail.com\u0026gt; George \u0026lt;ruan3@illinois.edu\u0026gt; David Levering \u0026lt;dmlevering@gmail.com\u0026gt; Bernard Lim \u0026lt;bernlim93@users.noreply.github.com\u0026gt; zwang180 \u0026lt;zshwang0809@gmail.com\u0026gt; xuanwang91 \u0026lt;LilyBiology2010@gmail.com\u0026gt; xin-0 \u0026lt;xintong2@illinois.edu\u0026gt; wchill \u0026lt;wchill1337@gmail.com\u0026gt; vishnui \u0026lt;vishnui@gmail.com\u0026gt; tvarun2013 \u0026lt;tvarun2013@gmail.com\u0026gt; sstevenshang \u0026lt;sstevenshang@users.noreply.github.com\u0026gt; ssquirrel \u0026lt;lxl_zhang@Hotmail.com\u0026gt; smeenai \u0026lt;shoaib.meenai@gmail.com\u0026gt; shrujancheruku \u0026lt;shrujancheruku@gmail.com\u0026gt; ruiqili2 \u0026lt;ruiqili2@users.noreply.github.com\u0026gt; rchwlsk2 \u0026lt;rchwlsk2@illinois.edu\u0026gt; ralphchung \u0026lt;ralphchung2005@gmail.com\u0026gt; nikioftime \u0026lt;ncwells2@illinois.edu\u0026gt; mosaic0123 \u0026lt;truffer@live.com\u0026gt; majiasheng \u0026lt;jiasheng.ma@yahoo.com\u0026gt; m \u0026lt;cheonghiuwaa@gmail.com\u0026gt; li820970 \u0026lt;li820970@gmail.com\u0026gt; kuck1 \u0026lt;kuck1@illinois.edu\u0026gt; kkgomez2 \u0026lt;kkgomez2@users.noreply.github.com\u0026gt; jjames34 \u0026lt;James_Jerry1@yahoo.com\u0026gt; jargals2 \u0026lt;jargals2@ilinois.edu\u0026gt; hzding621 \u0026lt;hzding621@users.noreply.github.com\u0026gt; hzding621 \u0026lt;hzding621@gmail.com\u0026gt; hsingh23 \u0026lt;hisingh1@gmail.com\u0026gt; denisdemaisbr \u0026lt;denis@roo.com.br\u0026gt; daishengliang \u0026lt;daishengliang@gmail.com\u0026gt; cucumbur \u0026lt;bomblolism@gmail.com\u0026gt; codechao999 \u0026lt;brianweis@comcast.net\u0026gt; chrisshroba \u0026lt;chrisshroba@gmail.com\u0026gt; cesarcastmore \u0026lt;cesar.cast.more@gmail.com\u0026gt; briantruong777 \u0026lt;briantruong777@users.noreply.github.com\u0026gt; botengHY \u0026lt;tengbo1992@gmail.com\u0026gt; blapalp \u0026lt;pzkmmmh@gmail.com\u0026gt; bchhetri1 \u0026lt;bhaweshchhetri1@gmail.com\u0026gt; anadella96 \u0026lt;aisha.nadella@gmail.com\u0026gt; akleiner2 \u0026lt;akleiner24@gmail.com\u0026gt; aRatnam12 \u0026lt;ansh.ratnam@gmail.com\u0026gt; Yash Sharma \u0026lt;yashosharma@gmail.com\u0026gt; Xiangbin Hu \u0026lt;xhu27@illinois.edu\u0026gt; WininWin \u0026lt;ezoneid@gmail.com\u0026gt; William Klock \u0026lt;william.klock@gmail.com\u0026gt; WenhanZ \u0026lt;marinebluee@hotmail.com\u0026gt; Vivek Pandya \u0026lt;vivekvpandya@gmail.com\u0026gt; Vineeth Puli \u0026lt;vpuli98@gmail.com\u0026gt; Vangelis Tsiatsianas \u0026lt;vangelists@users.noreply.github.com\u0026gt; Vadiml1024 \u0026lt;vadim@mbdsys.com\u0026gt; Utsav2 \u0026lt;ukshah2@illinois.edu\u0026gt; Thirumal Venkat \u0026lt;zapstar@users.noreply.github.com\u0026gt; TheEntangledLight \u0026lt;bdelapor@illinois.edu\u0026gt; SudarshanGp \u0026lt;SudarshanGp@users.noreply.github.com\u0026gt; Sudarshan Konge \u0026lt;6025419+sudk1896@users.noreply.github.com\u0026gt; Slix \u0026lt;slixpk@gmail.com\u0026gt; Sasank Chundi \u0026lt;sasank.chundi@gmail.com\u0026gt; SachinRaghunathan \u0026lt;srghnth2@illinois.edu\u0026gt; Rémy Léone \u0026lt;remy.leone@gmail.com\u0026gt; RusselLuo \u0026lt;russelluo@gmail.com\u0026gt; Roman Vaivod \u0026lt;littlewhywhat@gmail.com\u0026gt; Rohit Sarathy \u0026lt;rohit@sarathy.org\u0026gt; Rick Sheahan \u0026lt;bomblolism@gmail.com\u0026gt; Rakhim Davletkaliyev \u0026lt;freetonik@gmail.com\u0026gt; Punitvara \u0026lt;punitvara@gmail.com\u0026gt; Phillip Quy Le \u0026lt;pitlv2109@gmail.com\u0026gt; Pavle Simonovic \u0026lt;simonov2@illinois.edu\u0026gt; Paul Hindt \u0026lt;phindt@gmail.com\u0026gt; Nishant Maniam \u0026lt;nishant.maniam@gmail.com\u0026gt; Mustafa Altun \u0026lt;gmail@mustafaaltun.com\u0026gt; Mohammed Sadik P. K \u0026lt;sadiqpkp@gmail.com\u0026gt; Mingchao Zhang \u0026lt;43462732+mingchao-zhang@users.noreply.github.com\u0026gt; Michael Vanderwater \u0026lt;vndrwtr2@users.noreply.github.com\u0026gt; Maxiwell Luo \u0026lt;maxluoXIII@gmail.com\u0026gt; LunaMystic \u0026lt;suxianghan@outlook.com\u0026gt; Liam Monahan \u0026lt;liam@liammonahan.com\u0026gt; Joshua Wertheim \u0026lt;joshwertheim@gmail.com\u0026gt; John Pham \u0026lt;newhope11134@gmail.com\u0026gt; Johannes Scheuermann \u0026lt;johscheuer@users.noreply.github.com\u0026gt; Joey Bloom \u0026lt;15joeybloom@users.noreply.github.com\u0026gt; Jimmy Zhang \u0026lt;midnight.vivian@gmail.com\u0026gt; Jeffrey Foster \u0026lt;jmfoste2@illinois.edu\u0026gt; James Daniel \u0026lt;james-daniel@users.noreply.github.com\u0026gt; Jake Bailey \u0026lt;zikaeroh@gmail.com\u0026gt; JACKHAHA363 \u0026lt;luyuchen.paul@gmail.com\u0026gt; Hydrosis \u0026lt;badda2k@gmail.com\u0026gt; Hong \u0026lt;plantvsbird@gmail.com\u0026gt; Grant Wu \u0026lt;grantwu2@gmail.com\u0026gt; EvanFabry \u0026lt;Evan.Fabry@gmail.com\u0026gt; EddieVilla \u0026lt;EddieVilla@users.noreply.github.com\u0026gt; Deepak Nagaraj \u0026lt;n.deepak@gmail.com\u0026gt; Daniel Meir Doron \u0026lt;danielmeirdoron@gmail.com\u0026gt; Daniel Le \u0026lt;GreenRecycleBin@gmail.com\u0026gt; Daniel Jamrozik \u0026lt;djamro2@illinois.edu\u0026gt; Daniel Carballal \u0026lt;danielenriquecarballal@gmail.com\u0026gt; Daniel \u0026lt;DTV96Calibre@users.noreply.github.com\u0026gt; Daeyun Shin \u0026lt;daeyun@daeyunshin.com\u0026gt; Creyslz \u0026lt;creyslz@gmail.com\u0026gt; Christian Cygnus \u0026lt;gamer00@att.net\u0026gt; CharlieMartell \u0026lt;charliecmartell@gmail.com\u0026gt; Caleb Bassi \u0026lt;calebjbassi@gmail.com\u0026gt; Brian Kurek \u0026lt;brkurek@gmail.com\u0026gt; Brendan Wilson \u0026lt;brendan.x.wilson@gmail.com\u0026gt; Bo Liu \u0026lt;boliu1@illinois.edu\u0026gt; Ayush Ranjan \u0026lt;ayushr2@illinois.edu\u0026gt; Atul kumar Agrawal \u0026lt;ms.atul1303@gmail.com\u0026gt; Artur Sak \u0026lt;artursak1981@gmail.com\u0026gt; Ankush Agarwal \u0026lt;ankushagarwal@users.noreply.github.com\u0026gt; Angelino \u0026lt;angelino_m@outlook.com\u0026gt; Andrey Zaytsev \u0026lt;andzaytsev@gmail.com\u0026gt; Alex Yang \u0026lt;alyx.yang@gmail.com\u0026gt; Alex Cusack \u0026lt;cusackalex@gmail.com\u0026gt; Aidan Epstein \u0026lt;aidan@jmad.org\u0026gt; Ace Nassri \u0026lt;ace.nassri@gmail.com\u0026gt; Abdullahi Abdalla \u0026lt;abdalla6@illinois.edu\u0026gt; Aneesh Durg \u0026lt;durg2@illinois.edu\u0026gt; Assassin Eclipse \u0026lt;hungwoei96@hotmail.com\u0026gt; Eric Cao \u0026lt;eric7252000@gmail.com\u0026gt; Raphael Long \u0026lt;rafilong42@gmail.com\u0026gt; WeiL \u0026lt;z920631580@gmail.com\u0026gt; williamsentosa95 \u0026lt;38774380+williamsentosa95@users.noreply.github.com\u0026gt; Pradyumna Shome \u0026lt;pradyumna.shome@gmail.com\u0026gt; Benjamin West Pollak \u0026lt;benjaminwpollak@gmail.com\u0026gt; 姜芃越 Pengyue Jiang \u0026lt;pengyue3@illinois.edu\u0026gt; "},{"id":1,"href":"/cs341/en/chap2/","title":"Background","section":"Preface","content":" Background # Sometimes the journey of a thousand steps begins by learning to walk "},{"id":2,"href":"/cs341/en/chap7/Condition_Variables/","title":"Condition Variables","section":"Synchronization","content":" Condition Variables # Condition variables allow a set of threads to sleep until woken up. The API allows either one or all threads to be woken up. If a program only wakes one thread, the operating system will decide which thread to wake up. Threads don’t wake threads other directly like by id. Instead, a thread ‘signal’s the condition variable, which then will wake up one (or all) threads that are sleeping inside the condition variable.\nCondition variables are also used with a mutex and with a loop, so when woken up they have to check a condition in a critical section. If a thread needs to be woken up outside of a critical section, there are other ways to do this in POSIX. Threads sleeping inside a condition variable are woken up by calling pthread_cond_broadcast (wake up all) or pthread_cond_signal (wake up one). Note despite the function name, this has nothing to do with POSIX signals!\nOccasionally, a waiting thread may appear to wake up for no reason. This is called a spurious wakeup. If you read the hardware implementation of a mutex section, this is similar to the atomic failure of the same name.\nWhy do spurious wakeups happen? For performance. On multi-CPU systems, it is possible that a race condition could cause a wake-up (signal) request to be unnoticed. The kernel may not detect this lost wake-up call but can detect when it might occur. To avoid the potentially lost signal, the thread is woken up so that the program code can test the condition again. If you want to know why, check out the appendix.\n"},{"id":3,"href":"/cs341/en/chap2/Debugging_Environments/","title":"Debugging and Environments","section":"Background","content":" Debugging and Environments # I’m going to tell you a secret about this course: it is about working smarter not harder. The course can be time-consuming but the reason that so many people see it as such (and why so many students don’t see it as such) is the relative familiarity of people with their tools. Let’s go through some of the common tools that you’ll be working on and need to be familiar with.\nssh # ssh is short for the Secure Shell (“Ssh(1),” #ref-openbsd_ssh). It is a network protocol that allows you to spawn a shell on a remote machine. Most of the times in this class you will need to ssh into your VM like this\n$ ssh netid@sem-cs241-VM.cs.illinois.edu If you don’t want to type your password out every time, you can generate an ssh key that uniquely identifies your machine. If you already have a key pair, you can skip to the copy id stage.\n\u0026gt; ssh-keygen -t rsa -b 4096 # Do whatever keygen tells you # Don\u0026#39;t feel like you need a passcode if your login password is secure \u0026gt; ssh-copy-id netid@sem-cs241-VM.cs.illinois.edu # Enter your password for maybe the final time \u0026gt; ssh netid@sem-cs241-VM.cs.illinois.edu If you still think that that is too much typing, you can always alias hosts. You may need to restart your VM or reload sshd for this to take effect. The config file is available on Linux and Mac distros. For Windows, you’ll have to use the Windows Linux Subsystem or configure any aliases in PuTTY\n\u0026gt; cat ~/.ssh/config Host vm User netid HostName sem-cs241-VM.cs.illinois.edu \u0026gt; ssh vm git # What is ‘git‘? Git is a version control system. What that means is git stores the entire history of a directory. We refer to the directory as a repository. So what do you need to know is a few things. First, create your repository with the repo creator. If you haven’t already signed into enterprise GitHub, make sure to do so otherwise your repository won’t be created for you. After that, that means your repository is created on the server. Git is a decentralized version control system, meaning that you’ll need to get a repository onto your VM. We can do this with a clone. Whatever you do, do not go through the README.md tutorial.\n$ git clone https://github-dev.cs.illinois.edu/cs241-fa18/\u0026lt;netid\u0026gt;.git This will create a local repository. The workflow is you make a change on your local repository, add the changes to a current commit, actually commit, and push the changes to the server.\n$ # edit the file, maybe using vim $ git add \u0026lt;file\u0026gt; $ git commit -m \u0026#34;Committing my file\u0026#34; $ git push origin master Now to explain git well, you need to understand that git for our purposes will look like a linked list. You will always be at the head of master, and you will do the edit-add-commit-push loop. We have a separate branch on Github that we push feedback to under a specific branch which you can view on Github website. The markdown file will have information on the test cases and results (like standard out).\nEvery so often git can break. Here is a list of commands you probably won’t need to fix your repo\ngit-cherry-pick git-pack git-gc git-clean git-rebase git-stash/git-apply/git-pop git-branch If you are currently on a branch, and you don’t see either\n$ git status On branch master Your branch is up-to-date with \u0026#39;origin/master\u0026#39;. nothing to commit, working directory clean or\n$ git status On branch master Your branch is up-to-date with \u0026#39;origin/master\u0026#39;. Changes not staged for commit: (use \u0026#34;git add \u0026lt;file\u0026gt;...\u0026#34; to update what will be committed) (use \u0026#34;git checkout -- \u0026lt;file\u0026gt;...\u0026#34; to discard changes in working directory) modified: \u0026lt;FILE\u0026gt; ... no changes added to commit (use \u0026#34;git add\u0026#34; and/or \u0026#34;git commit -a\u0026#34;) And something like\n$ git status HEAD detached at 4bc4426 nothing to commit, working directory clean Don’t panic, but your repository may be in an unworkable state. If you aren’t nearing a deadline, come to office hours or ask your question on Piazza, and we’d be happy to help. In an emergency scenario, delete your repository and re-clone (you’ll have to add the release as above). This will lose any local uncommitted changes. Make sure to copy any files you were working on outside the directory, remove and copy them back in\nIf you want to learn more about git, there are all but an endless number of tutorials and resources online that can help you. Here are some links that can help you out\nhttps://git-scm.com/docs/gittutorial\nhttps://www.atlassian.com/git/tutorials/what-is-version-control\nhttps://thenewstack.io/tutorial-git-for-absolutely-everyone/\nEditors # Some people take this as an opportunity to learn a new editor, others not so much. The first part is those of you who want to learn a new editor. In the editor war that spans decades, we have come to the battle of vim vs emacs.\nVim is a text editor and a Unix-like utility. You enter vim by typing vim [file], which takes you into the editor. There are three most commonly used modes: normal mode, insert mode, and command mode. You start off in normal mode. In this mode, you can move around with many keys with the most common ones being hjkl (corresponding to left, down, up, and right respectively). To run commands in vim, you can first type : and then a command after it. For instance, to quit vim, simply type :q (q stands for quit). If you have any unsaved edits, you must either save them :w, save and quit :wq, or quit and discard changes :q!. To make edits you can either type i to change you into insert mode or a to change to insert mode after the cursor. This is the basics when it comes to vim. In addition to the countless great resources out there on the internet, vim also has its own built-in tutorials set up for beginners. To access the interactive tutorial, enter vimtutor in the command line (not inside of vim), and you are all set!\nEmacs is more of a way of life, and I don’t mean that figuratively. A lot of people say that emacs is a powerful operating system lacking a decent text editor. This means emacs can house a terminal, gdb session, ssh session, code and a whole lot more. It would not be fitting any other way to introduce you to the gnu-emacs any other way than the gnu-docs https://www.gnu.org/software/emacs/tour/. Just note that emacs is insanely powerful. You can do almost anything with it. There are a fair number of students who like the IDE-aspect of other programming languages. Know that you can set up emacs to be an IDE, but you have to learn a bit of Lisp http://martinsosic.com/development/emacs/2017/12/09/emacs-cpp-ide.html.\nThen there are those of you who like to use your own editors. That is completely fine. For this, we require sshfs which has ports on many different machines.\nWindows https://github.com/billziss-gh/sshfs-win\nMac https://github.com/osxfuse/osxfuse/wiki/SSHFS\nLinux https://help.ubuntu.com/community/SSHFS\nAt that point, the files on your VM are synced with the files on your machine and edits can be made and will be synced.\nAt the time of writing, an author likes to use spacemacs http://spacemacs.org/ which marries both vim and emacs and both of their difficulties. I’ll give my soapbox for why I like it, but be warned that if you are starting from absolutely no vim or emacs experience the learning curve along with this course may be too much.\nExtensible. Spacemacs has a clean design written in lisp. There are 100s of packages ready to be installed by editing your spacemacs config and reloading that do everything from syntax checking, automatic static analyzing, etc.\nMost of the good parts from vim and emacs. Emacs is good at doing everything by being a fast editor. Vim is good at making fast edits and moving around. Spacemacs is the best of both worlds allowing vim keybindings to all the emacs goodness underneath.\nLots of preconfiguration done. As opposed with a fresh emacs install, a lot of the configurations with language and projects are done for you like neotree, helm, various language layers. All that you have to do is navigate neotree to the base of your project and emacs will turn into an IDE for that programming language.\nBut obviously to each his or her own. Many people will argue that editor gurus spend more time editing their editors and actually editing.\nClean Code # Make your code modular using helper functions. If there is a repeated task (getting the pointers to contiguous blocks in the malloc MP, for example), make them helper functions. And make sure each function does one thing well so that you don’t have to debug twice. Let’s say that we are doing selection sort by finding the minimum element each iteration like so,\nvoid selection_sort(int *a, long len){ for(long i = len-1; i \u0026gt; 0; --i){ long max_index = i; for(long j = len-1; j \u0026gt;= 0; --j){ if(a[max_index] \u0026lt; a[j]){ max_index = j; } } int temp = a[i]; a[i] = a[max_index]; a[max_index] = temp; } } Many can see the bug in the code, but it can help to refactor the above method into\nlong max_index(int *a, long start, long end); void swap(int *a, long idx1, long idx2); void selection_sort(int *a, long len); And the error is specifically in one function. In the end, this class is about writing system programs, not a class about refactoring/debugging your code. In fact, most kernel code is so atrocious that you don’t want to read it – the defense there is that it needs to be. But for the sake of debugging, it may benefit you in the long run to adopt some of these practices.\nAsserts # Use assertions to make sure your code works up to a certain point – and importantly, to make sure you don’t break it later. For example, if your data structure is a doubly-linked list, you can do something like assert(node == node-\u0026gt;next-\u0026gt;prev) to assert that the next node has a pointer to the current node. You can also check the pointer is pointing to an expected range of memory address, non-null, -\u0026gt;size is reasonable, etc. The DEBUG macro will disable all assertions, so don’t forget to set that once you finish debugging (“Assert,”1).\nHere is a quick example with an assert. Let’s say that we are writing code using memcpy. We would want to put an assert before that checks whether my two memory regions overlap. If they do overlap, memcpy runs into undefined behavior, so we want to catch that problem than later.\nassert(!(src \u0026lt; dest+n \u0026amp;\u0026amp; dest \u0026lt; src+n)); //Checks overlap memcpy(dest, src, n);\nThis check can be turned off at compile-time, but will save you tons of trouble debugging!\n“Assert.” n.d. Cplusplus.com. cplusplus.com. http://www.cplusplus.com/reference/cassert/assert/.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n"},{"id":4,"href":"/cs341/en/chap2/GDB/","title":"GDB","section":"Background","content":" GDB # GDB is short for the GNU Debugger. GDB is a program that helps you track down errors by interactively debugging them (“GDB: The Gnu Project Debugger”1). It can start and stop your program, look around, and put in ad hoc constraints and checks. Here are a few examples.\nSetting breakpoints programmatically # A breakpoint is a line of code where you want the execution to stop and give control back to the debugger. A useful trick when debugging complex C programs with GDB is setting breakpoints in the source code.\nint main() { int val = 1; val = 42; asm(\u0026#34;int $3\u0026#34;); // set a breakpoint here val = 7; } $ gcc main.c -g -o main $ gdb --args ./main (gdb) r [...] Program received signal SIGTRAP, Trace/breakpoint trap. main () at main.c:6 6 val = 7; (gdb) p val $1 = 42 You can also set breakpoints programmatically. Assume that we have no optimization and the line numbers are as follows\nint main() { int val = 1; val = 42; val = 7; } We can now set the breakpoint before the program starts.\n$ gcc main.c -g -o main $ gdb --args ./main (gdb) break main.c:4 [...] (gdb) p val $1 = 42 Checking memory content # We can also use gdb to check the content of different pieces of memory. For example,\nint main() { char bad_string[3] = {\u0026#39;C\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;t\u0026#39;}; printf(\u0026#34;%s\u0026#34;, bad_string); } Compiled we get\n$ gcc main.c -g -o main \u0026amp;\u0026amp; ./main $ Cat ZVQ<56> $ We can now use gdb to look at specific bytes of the string and reason about when the program should’ve stopped running\n(gdb) l 1 #include \u0026lt;stdio.h\u0026gt; 2 int main() { 3 char bad_string[3] = {\u0026#39;C\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;t\u0026#39;}; 4 printf(\u0026#34;%s\u0026#34;, bad_string); 5 } (gdb) b 4 Breakpoint 1 at 0x100000f57: file main.c, line 4. (gdb) r [...] Breakpoint 1, main () at main.c:4 4 printf(\u0026#34;%s\u0026#34;, bad_string); (gdb) x/16xb bad_string 0x7fff5fbff9cd: 0x63 0x61 0x74 0xe0 0xf9 0xbf 0x5f 0xff 0x7fff5fbff9d5: 0x7f 0x00 0x00 0xfd 0xb5 0x23 0x89 0xff (gdb) Here, by using the x command with parameters 16xb, we can see that starting at memory address 0x7fff5fbff9c`` (value of bad_string), printf` would actually see the following sequence of bytes as a string because we provided a malformed string without a null terminator.\nInvolved gdb example # Here is how one of your TAs would go through and debug a simple program that is going wrong. First, the source code of the program. If you can see the error immediately, please bear with us.\n#include \u0026lt;stdio.h\u0026gt; double convert_to_radians(int deg); int main(){ for (int deg = 0; deg \u0026gt; 360; ++deg){ double radians = convert_to_radians(deg); printf(\u0026#34;%d. %f\\n\u0026#34;, deg, radians); } return 0; } double convert_to_radians(int deg){ return ( 31415 / 1000 ) * deg / 180; } How can we use gdb to debug? First we ought to load GDB.\n$ gdb --args ./main (gdb) layout src; # If you want a GUI type (gdb) run (gdb) Want to take a look at the source?\n(gdb) l 1 #include \u0026lt;stdio.h\u0026gt; 2 3 double convert_to_radians(int deg); 4 5 int main(){ 6 for (int deg = 0; deg \u0026gt; 360; ++deg){ 7 double radians = convert_to_radians(deg); 8 printf(\u0026#34;%d. %f\\n\u0026#34;, deg, radians); 9 } 10 return 0; (gdb) break 7 # break \u0026lt;file\u0026gt;:line or break \u0026lt;file\u0026gt;:function (gdb) run (gdb) From running the code, the breakpoint didn’t even trigger, meaning the code never got to that point. That’s because of the comparison! Okay, flip the sign it should work now right?\n(gdb) run 350. 60.000000 351. 60.000000 352. 60.000000 353. 60.000000 354. 60.000000 355. 61.000000 356. 61.000000 357. 61.000000 358. 61.000000 359. 61.000000 (gdb) break 14 if deg == 359 # Let\u0026#39;s check the last iteration only (gdb) run ... (gdb) print/x deg # print the hex value of degree $1 = 0x167 (gdb) print (31415/1000) $2 = 0x31 (gdb) print (31415/1000.0) $3 = 201.749 (gdb) print (31415.0/10000.0) $4 = 3.1414999999999999 That was only the bare minimum, though most of you will get by with that. There are a whole load more resources on the web, here are a few specific ones that can help you get started.\nhttp://www.cs.cmu.edu/~gilpin/tutorial/\nhttp://www.delorie.com/gnu/docs/gdb/gdb_56.html\nhttps://www.youtube.com/watch?v=PorfLSr3DDI\nShell # What do you actually use to run your program? A shell! A shell is a programming language that is running inside your terminal. A terminal is merely a window to input commands. Now, on POSIX we usually have one shell called sh that is linked to a POSIX compliant shell called dash. Most of the time, you use a shell called bash that is somewhat POSIX compliant but has some nifty built-in features. If you want to be even more advanced, zsh has some more powerful features like tab complete on programs and fuzzy patterns.\nUndefined Behavior Sanitizer # The undefined behavior sanitizer is a wonderful tool provided by the llvm project. It allows you to compile code with a runtime checker to make sure that you don’t do undefined behavior for various categories. We will try to include it into our projects, but requires support form all the external libraries that we use so we may not get around to all of them. https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html\nUndefined behavior - why we can’t solve it in general\nAlso please please read Chris Lattner’s 3 Part blog post on undefined behavior. It can shed light on debug builds and the mystery of compiler optimization.\nhttp://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html\nClang Static Build Tools # Clang provides a great drop-in replacement tools for compiling programs. If you want to see if there is an error that may cause a race condition, casting error, etc, all you need to do is the following.\n$ scan-build make And in addition to the make output, you will get static build warnings.\nstrace and ltrace # strace and ltrace are two programs that trace the system calls and library calls respectively of a running program or command. These may be missing on your system so to install feel free to run the following.\n$ sudo apt install strace ltrace Debugging with ltrace can be as simple as figuring out what was the return call of the last library call that failed.\nint main() { FILE *fp = fopen(\u0026#34;I don\u0026#39;t exist\u0026#34;, \u0026#34;r\u0026#34;); fprintf(fp, \u0026#34;a\u0026#34;); fclose(fp); return 0; } \u0026gt; ltrace ./a.out __libc_start_main(0x8048454, 1, 0xbfc19db4, 0x80484c0, 0x8048530 \u0026lt;unfinished ...\u0026gt; fopen(\u0026#34;I don\u0026#39;t exist\u0026#34;, \u0026#34;r\u0026#34;) = 0x0 fwrite(\u0026#34;Invalid Write\\n\u0026#34;, 1, 14, 0x0 \u0026lt;unfinished ...\u0026gt; --- SIGSEGV (Segmentation fault) --- +++ killed by SIGSEGV +++ ltrace output can clue you in to weird things your program is doing live. Unfortunately, ltrace can’t be used to inject faults, meaning that ltrace can tell you what is happening, but it can’t tamper with what is already happening.\nstrace on the other hand could modify your program. Debugging with strace is amazing. The basic usage is running strace with a program, and it’ll get you a complete list of system call parameters.\n$ strace head README.md execve(\u0026#34;/usr/bin/head\u0026#34;, [\u0026#34;head\u0026#34;, \u0026#34;README.md\u0026#34;], 0x7ffff28c8fa8 /* 60 vars */) = 0 brk(NULL) = 0x7ffff5719000 access(\u0026#34;/etc/ld.so.nohwcap\u0026#34;, F_OK) = -1 ENOENT (No such file or directory) access(\u0026#34;/etc/ld.so.preload\u0026#34;, R_OK) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, \u0026#34;/etc/ld.so.cache\u0026#34;, O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=32804, ...}) = 0 ... If the output is too verbose, you can use trace= with a command delimited list of syscalls to filter all but those calls.\n$ strace -e trace=read,write head README.md read(3, \u0026#34;\\177ELF\\2\\1\\1\\3\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0\u0026gt;\\0\\1\\0\\0\\0\\260\\34\\2\\0\\0\\0\\0\\0\u0026#34;..., 832) = 832 read(3, \u0026#34;# Locale name alias data base.\\n#\u0026#34;..., 4096) = 2995 read(3, \u0026#34;\u0026#34;, 4096) = 0 read(3, \u0026#34;# C Datastructures\\n\\n[![Build Sta\u0026#34;..., 8192) = 1250 write(1, \u0026#34;# C Datastructures\\n\u0026#34;, 19# C Datastructures You can also trace files or targets.\n$ strace -e trace=read,write -P README.md head README.md strace: Requested path \u0026#39;README.md\u0026#39; resolved into \u0026#39;/mnt/c/Users/user/personal/libds/README.md\u0026#39; read(3, \u0026#34;# C Datastructures\\n\\n[![Build Sta\u0026#34;..., 8192) = 1250 Newer versions of strace can actually inject faults into your program. This is useful when you want to occasionally make reads and writes fail for example in a networking application, which your program should handle. The problem is as of early 2019, that version is missing from Ubuntu repositories. Meaning that you’ll have to install it from the source.\nprintfs # When all else fails, print! Each of your functions should have an idea of what it is going to do. You want to test that each of your functions is doing what it set out to do and see exactly where your code breaks. In the case with race conditions, tsan may be able to help, but having each thread print out data at certain times could help you identify the race condition.\nTo make printfs useful, try to have a macro that fills in the context by which the printf was called – a log statement if you will. A simple useful but untested log statement could be as follows. Try to make a test and figure out something that is going wrong, then log the state of your variables.\n#include \u0026lt;execinfo.h\u0026gt; #include \u0026lt;stdio.h\u0026gt; #include \u0026lt;stdlib.h\u0026gt; #include \u0026lt;stdarg.h\u0026gt; #include \u0026lt;unistd.h\u0026gt; // bt is print backtrace const int num_stack = 10; int __log(int line, const char *file, int bt, const char *fmt, ...) { if (bt) { void *raw_trace[num_stack]; size_t size = backtrace(raw_trace, sizeof(raw_trace) / sizeof(raw_trace[0])); char **syms = backtrace_symbols(raw_trace, size); for(ssize_t i = 0; i \u0026lt; size; i++) { fprintf(stderr, \u0026#34;|%s:%d| %s\\n\u0026#34;, file, line, syms[i]); } free(syms); } int ret = fprintf(stderr, \u0026#34;|%s:%d| \u0026#34;, file, line); va_list args; va_start(args, fmt); ret += vfprintf(stderr, fmt, args); va_end(args); ret += fprintf(stderr, \u0026#34;\\n\u0026#34;); return ret; } #ifdef DEBUG #define log(...) __log(__LINE__, __FILE__, 0, __VA_ARGS__) #define bt(...) __log(__LINE__, __FILE__, 1, __VA_ARGS__) #else #define log(...) #define bt(...) #endif //Use as log(args like printf) or bt(args like printf) to either log or get backtrace int main() { log(\u0026#34;Hello Log\u0026#34;); bt(\u0026#34;Hello Backtrace\u0026#34;); } And then use as appropriately. Check out the compiling and linking section in the appendix if you have any questions on how a C program gets translated to machine code.\n“GDB: The Gnu Project Debugger.” 2019. GDB: The GNU Project Debugger. Free Software Foundation. https://www.gnu.org/software/gdb/.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n"},{"id":5,"href":"/cs341/en/chap2/Homework0/","title":"Homework 0","section":"Background","content":" Homework 0 # // First, can you guess which lyrics have been transformed into this C-like system code? char q[] = \u0026#34;Do you wanna build a C99 program?\u0026#34;; #define or \u0026#34;go debugging with gdb?\u0026#34; static unsigned int i = sizeof(or) != strlen(or); char* ptr = \u0026#34;lathe\u0026#34;; size_t come = fprintf(stdout,\u0026#34;%s door\u0026#34;, ptr+2); int away = ! (int) * \u0026#34;\u0026#34;; int* shared = mmap(NULL, sizeof(int*), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); munmap(shared,sizeof(int*)); if(!fork()) { execlp(\u0026#34;man\u0026#34;,\u0026#34;man\u0026#34;,\u0026#34;-3\u0026#34;,\u0026#34;ftell\u0026#34;, (char*)0); perror(\u0026#34;failed\u0026#34;); } if(!fork()) { execlp(\u0026#34;make\u0026#34;,\u0026#34;make\u0026#34;, \u0026#34;snowman\u0026#34;, (char*)0); execlp(\u0026#34;make\u0026#34;,\u0026#34;make\u0026#34;, (char*)0)); } exit(0); So you want to master System Programming? And get a better grade than B?\nint main(int argc, char** argv) { puts(\u0026#34;Great! We have plenty of useful resources for you, but it\u0026#39;s up to you to\u0026#34;); puts(\u0026#34; be an active learner and learn how to solve problems and debug code.\u0026#34;); puts(\u0026#34;Bring your near-completed answers the problems below\u0026#34;); puts(\u0026#34; to the first lab to show that you\u0026#39;ve been working on this.\u0026#34;); printf(\u0026#34;A few \\\u0026#34;don\u0026#39;t knows\\\u0026#34; or \\\u0026#34;unsure\\\u0026#34; is fine for lab 1.\\n\u0026#34;); puts(\u0026#34;Warning: you and your peers will work hard in this class.\u0026#34;); puts(\u0026#34;This is not CS225; you will be pushed much harder to\u0026#34;); puts(\u0026#34; work things out on your own.\u0026#34;); fprintf(stdout,\u0026#34;This homework is a stepping stone to all future assignments.\\n\u0026#34;); char p[] = \u0026#34;So, you will want to clear up any confusions or misconceptions.\\n\u0026#34;; write(1, p, strlen(p) ); char buffer[1024]; sprintf(buffer,\u0026#34;For grading purposes, this homework 0 will be graded as part of your lab %d work.\\n\u0026#34;, 1); write(1, buffer, strlen(buffer)); printf(\u0026#34;Press Return to continue\\n\u0026#34;); read(0, buffer, sizeof(buffer)); return 0; } Watch the videos and write up your answers to the following questions\nImportant!\nThe virtual machine-in-your-browser and the videos you need for HW0 are here:\nhttp://cs-education.github.io/sys/\nQuestions? Comments? Use the current semester’s CS241 Piazza: https://piazza.com/\nThe in-browser virtual machine runs entirely in JavaScript and is fastest in Chrome. Note the VM and any code you write is reset when you reload the page, so copy your code to a separate document. The post-video challenges are not part of homework 0, but you learn the most by doing rather than passively watching. You have some fun with each end-of-video challenge.\nHW0 questions are below. Copy your answers into a text document because you’ll need to submit them later in the course.\nChapter 1 # In which our intrepid hero battles standard out, standard error, file descriptors and writing to files\nHello, World! (system call style) Write a program that uses write() to print out “Hi! My name is ”.\nHello, Standard Error Stream! Write a function to print out a triangle of height n to standard error. Your function should have the signature void write_triangle(int n) and should use write(). The triangle should look like this, for n = 3:\n**\nWriting to files Take your program from “Hello, World!” modify it write to a file called hello_world.txt. Make sure to use correct flags and a correct mode for open() (man 2 open is your friend).\nNot everything is a system call Take your program from “Writing to files” and replace write() with printf(). Make sure to print to the file instead of standard out!\nWhat are some differences between write() and printf()?\nChapter 2 # Sizing up C types and their limits, int and char arrays, and incrementing pointers\nHow many bits are there in a byte?\nHow many bytes are there in a char?\nHow many bytes the following are on your machine? int, double, float, long, and long long\nOn a machine with 8 byte integers, the declaration for the variable data is int data[8]. If the address of data is 0x7fbd9d40, then what is the address of data+2?\nWhat is data[3] equivalent to in C? Hint: what does C convert data[3] to before dereferencing the address? Remember, the type of a string constant \u0026ldquo;abc\u0026rdquo; is an array.\nWhy does this SEGFAULT?\nchar *ptr = \u0026#34;hello\u0026#34;; *ptr = \u0026#39;J\u0026#39;; What is the value of the variable str_size?\nssize_t str_size = sizeof(\u0026#34;Hello\\0World\u0026#34;) What is the value of the variable str_len\nssize_t str_len = strlen(\u0026#34;Hello\\0World\u0026#34;) Give an example of X such that sizeof(X) is 3.\nGive an example of Y such that sizeof(Y) might be 4 or 8 depending on the machine.\nChapter 3 # Program arguments, environment variables, and working with character arrays (strings)\nWhat are at least two ways to find the length of argv?\nWhat does argv[0] represent?\nWhere are the pointers to environment variables stored (on the stack, the heap, somewhere else)?\nOn a machine where pointers are 8 bytes, and with the following code:\nchar *ptr = \u0026#34;Hello\u0026#34;; char array[] = \u0026#34;Hello\u0026#34;; What are the values of sizeof(ptr) and sizeof(array)? Why?\nWhat data structure manages the lifetime of automatic variables?\nChapter 4 # Heap and stack memory, and working with structs\nIf I want to use data after the lifetime of the function it was created in ends, where should I put it? How do I put it there?\nWhat are the differences between heap and stack memory?\nAre there other kinds of memory in a process?\nFill in the blank: “In a good C program, for every malloc, there is a ___”.\nWhat is one reason malloc can fail?\nWhat are some differences between time() and ctime()?\nWhat is wrong with this code snippet?\nfree(ptr); free(ptr); What is wrong with this code snippet?\nfree(ptr); printf(\u0026#34;%s\\n\u0026#34;, ptr); How can one avoid the previous two mistakes?\nCreate a struct that represents a Person. Then make a typedef, so that struct Person can be replaced with a single word. A person should contain the following information: their name (a string), their age (an integer), and a list of their friends (stored as a pointer to an array of pointers to Persons).\nNow, make two persons on the heap, “Agent Smith” and “Sonny Moore”, who are 128 and 256 years old respectively and are friends with each other. Create functions to create and destroy a Person (Person’s and their names should live on the heap).\ncreate() should take a name and age. The name should be copied onto the heap. Use malloc to reserve sufficient memory for everyone having up to ten friends. Be sure initialize all fields (why?).\ndestroy() should free up both the memory of the person struct and all of its attributes that are stored on the heap. Destroying one person keeps other people in tact any other.\nChapter 5 # Text input and output and parsing using getchar, gets, and getline.\nWhat functions can be used for getting characters from stdin and writing them to stdout?\nName one issue with gets().\nWrite code that parses the string “Hello 5 World” and initializes 3 variables to “Hello”, 5, and “World”.\nWhat does one need to define before including getline()?\nWrite a C program to print out the content of a file line-by-line using getline().\nC Development # These are general tips for compiling and developing using a compiler and git. Some web searches will be useful here\nWhat compiler flag is used to generate a debug build?\nYou fix a problem in the Makefile and type make again. Explain why this may be insufficient to generate a new build.\nAre tabs or spaces used to indent the commands after the rule in a Makefile?\nWhat does git commit do? What’s a sha in the context of git?\nWhat does git log show you?\nWhat does git status tell you and how would the contents of .gitignore change its output?\nWhat does git push do? Why is it insufficient to commit with git commit -m ’fixed all bugs’?\nWhat does a non-fast-forward error git push reject mean? What is the most common way of dealing with this?\nOptional: Just for fun # Convert a song lyrics into System Programming and C code covered in this wiki book and share on Piazza.\nFind, in your opinion, the best and worst C code on the web and post the link to Piazza.\nWrite a short C program with a deliberate subtle C bug and post it on Piazza to see if others can spot your bug.\nDo you have any cool/disastrous system programming bugs you’ve heard about? Feel free to share with your peers and the course staff on Piazza.\n"},{"id":6,"href":"/cs341/en/chap1/","title":"Introduction","section":"Preface","content":" Introduction # To thy happy children of the future, those of the past send greetings. - Alma Mater At the University of Illinois at Urbana-Champaign, We fundamentally believe that we have a right to make the university better for all future students. It is a message etched into our Alma Mater and makes up the DNA of our course staff. As such, we created the coursebook. The coursebook is a free and open systems programming textbook that anyone can read, contribute to, and modify for now and forever. We don’t think information should be behind a walled garden, and we truly believe that complex concepts can be explained simply and fully, for anyone to understand. The goal of this book is to teach you the basics and give you some intuition into the complexities of systems programming.\nLike any good book, it isn’t complete. We still have plenty of examples, ideas, typos, and chapters to work on. If you find any issues, please file an https://github.com/illinois-cs241/coursebook/issues or email a list of typos to http://cs241.cs.illinois.edu/staff, and we’ll be happy to work on it. We are constantly trying to make the book better for students a year and ten years from now.\nThis work is based on the original coursebook located at https://github.com/angrave/SystemProgramming/wiki. All these peoples’ hard work is included in the section below.\nOh and the duck? Keep reading until synchronization :).\nThanks again and happy reading!\n– Bhuvy\n"},{"id":7,"href":"/cs341/en/chap5/","title":"Memory Allocators","section":"Preface","content":" Memory Allocators # Memory memory everywhere but not an allocation to be made - A fragmented heap "},{"id":8,"href":"/cs341/en/chap7/Mutex/","title":"Mutex","section":"Synchronization","content":" Mutex # Synchronization coordinates various tasks so that they all finishin the the correct state. In C, we have series of mechanisms to control what threads are allowed to perform at a given state. Most of the time, the threads can progress without having to communicate, but every so often two or more threads may want to access a critical section. A critical section is a section of code that can only be executed by one thread at a time if the program is to function correctly. If two threads (or processes) were to execute code inside the critical section at the same time, it is possible that the program may no longer have the correct behavior.\nAs we said in the previous chapter, race conditions happen when an operation touches a piece of memory at the same time as another thread. If the memory location is only accessible by one thread, for example the automatic variable i below, then there is no possibility of a race condition and no Critical Section associated with i. However, the sum variable is a global variable and accessed by two threads. It is possible that two threads may attempt to increment the variable at the same time.\n#include \u0026lt;stdio.h\u0026gt; #include \u0026lt;pthread.h\u0026gt; int sum = 0; //shared void *countgold(void *param) { int i; //local to each thread for (i = 0; i \u0026lt; 10000000; i++) { sum += 1; } return NULL; } int main() { pthread_t tid1, tid2; pthread_create(\u0026amp;tid1, NULL, countgold, NULL); pthread_create(\u0026amp;tid2, NULL, countgold, NULL); //Wait for both threads to finish: pthread_join(tid1, NULL); pthread_join(tid2, NULL); printf(\u0026#34;ARRRRG sum is %d\\n\u0026#34;, sum); return 0; } A typical output of the above code is ARGGGH sum is \u0026lt;some number less than expected\u0026gt; because there is a race condition. The code allows two threads to read and write sum at the same time. For example, both threads copy the current value of sum into CPU that runs each thread (let’s pick 123). Both threads increment one to their own copy. Both threads write back the value (124). If the threads had accessed the sum at different times then the count would have been 125. A few of the possible different orderings are below.\nPermissible Pattern:\nThread 1 Thread 2 Load Addr, Add 1 (i=1 locally) … Store (i=1 globally) … … Load Addr, Add 1 (i=2 locally) … Store (i=2 globally) Good Thread Access Pattern\nPartial Overlap:\nThread 1 Thread 2 Load Addr, Add 1 (i=1 locally) … Store (i=1 globally) Load Addr, Add 1 (i=1 locally) … Store (i=1 globally) Bad Thread Access Pattern\nFull Overlap\nThread 1 Thread 2 Load Addr, Add 1 (i=1 locally) Load Addr, Add 1 (i=1 locally) Store (i=1 globally) Store (i=1 globally) Horrible Thread Access Pattern\nWe would like the first pattern of the code being mutually exclusive. Which leads us to our first synchronization primitive, a Mutex.\nMutex # To ensure that only one thread at a time can access a global variable, use a mutex – short for Mutual Exclusion. If one thread is currently inside a critical section we would like another thread to wait until the first thread is complete. A mutex isn’t a primitive in the truest sense, though it is one of the smallest that has useful threading API. A mutex also isn’t a data structure. It is an abstract data type.\nLet’s think about a duck satisfying the mutex api. If someone has the duck then they are allowed to access a shared resource! We call it the mutex duck. Everyone else has to waddle around and wait. Once someone let’s go of the duck, they have to stop interacting with the resource and the next grabber can interact with the shared resource. Now you know the origins of the duck.\nThere are many ways to implement a mutex, and we’ll give a few in this chapter. For right now let’s use the black box that the pthread library gives us. Here is how we declare a mutex.\npthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER; // global variable pthread_mutex_lock(\u0026amp;m); // start of Critical Section // Critical section pthread_mutex_unlock(\u0026amp;m); //end of Critical Section Mutex Lifetime # There are a few ways of initializing a mutex. A program can use the macro PTHREAD_MUTEX_INITIALIZER only for global (‘static’) variables. m = PTHREAD_MUTEX_INITIALIZER is functionally equivalent to the more general purpose pthread_mutex_init(\u0026amp;m,NULL). The init version includes options to trade performance for additional error-checking and advanced sharing options. The init version also makes sure that the mutex is correctly initialized after the call, global mutexes are initialized on the first lock. A program can also call the init function inside of a program for a mutex located on the heap.\npthread_mutex_t *lock = malloc(sizeof(pthread_mutex_t)); pthread_mutex_init(lock, NULL); //later pthread_mutex_destroy(lock); free(lock); Once we are finished with the mutex we should also call pthread_mutex_destroy(\u0026amp;m) too. Note, a program can only destroy an unlocked mutex, destroy on a locked mutex is undefined behavior. Things to keep in mind about init and destroy A program doesn’t need to destroy a mutex created with the global initializer.\nMultiple threads init/destroy has undefined behavior\nDestroying a locked mutex has undefined behavior\nKeep to the pattern of one and only one thread initializing a mutex.\nCopying the bytes of the mutex to a new memory location and then using the copy is not supported. To reference a mutex, a program must to have a pointer to that memory address.\nMutex Usages # How does one use a mutex? Here is a complete example in the spirit of the earlier piece of code.\n#include \u0026lt;stdio.h\u0026gt; #include \u0026lt;pthread.h\u0026gt; // Create a mutex this ready to be locked! pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER; int sum = 0; void *countgold(void *param) { int i; //Same thread that locks the mutex must unlock it //Critical section is \u0026#39;sum += 1\u0026#39; //However locking and unlocking ten million times //has significant overhead pthread_mutex_lock(\u0026amp;m); // Other threads that call lock will have to wait until we call unlock for (i = 0; i \u0026lt; 10000000; i++) { sum += 1; } pthread_mutex_unlock(\u0026amp;m); return NULL; } int main() { pthread_t tid1, tid2; pthread_create(\u0026amp;tid1, NULL, countgold, NULL); pthread_create(\u0026amp;tid2, NULL, countgold, NULL); pthread_join(tid1, NULL); pthread_join(tid2, NULL); printf(\u0026#34;ARRRRG sum is %d\\n\u0026#34;, sum); return 0; } In the code above, the thread gets the lock to the counting house before entering. The critical section is only the sum+=1 so the following version is also correct.\nfor (i = 0; i \u0026lt; 10000000; i++) { pthread_mutex_lock(\u0026amp;m); sum += 1; pthread_mutex_unlock(\u0026amp;m); } return NULL; } This process runs slower because we lock and unlock the mutex a million times, which is expensive - at least compared with incrementing a variable. In this simple example, we didn’t need threads - we could have added up twice! A faster multi-thread example would be to add one million using an automatic (local) variable and only then adding it to a shared total after the calculation loop has finished:\nint local = 0; for (i = 0; i \u0026lt; 10000000; i++) { local += 1; } pthread_mutex_lock(\u0026amp;m); sum += local; pthread_mutex_unlock(\u0026amp;m); return NULL; } If you know the Gaussian sum, you can avoid race conditions altogether, but this is for illustration.\nStarting with the gotchas. Firstly, C Mutexes do not lock variables. A mutex is a simple data structure. It works with code, not data. If a mutex is locked, the other threads will continue. It’s only when a thread attempts to lock a mutex that is already locked, will the thread have to wait. As soon as the original thread unlocks the mutex, the second (waiting) thread will acquire the lock and be able to continue. The following code creates a mutex that does effectively nothing.\nint a; pthread_mutex_t m1 = PTHREAD_MUTEX_INITIALIZER, m2 = = PTHREAD_MUTEX_INITIALIZER; // later // Thread 1 pthread_mutex_lock(\u0026amp;m1); a++; pthread_mutex_unlock(\u0026amp;m1); // Thread 2 pthread_mutex_lock(\u0026amp;m2); a++; pthread_mutex_unlock(\u0026amp;m2); Here are some other gotchas in no particular order\nDon’t cross the streams! If using threads, don’t fork in the middle of your program. This means any time after your mutexes have been initialized.\nThe thread that locks a mutex is the only thread that can unlock it.\nEach program can have multiple mutex locks. A thread safe design might include a lock with each data structure, one lock per heap, or one lock per set of data structures If a program has only one lock, then there may be significant contention for the lock. If two threads were updating two different counters, it isn’t necessary to use the same lock.\nLocks are only tools. They don’t spot critical sections!\nThere will always be a small amount of overhead of calling pthread_mutex_lock and pthread_mutex_unlock. However, this is the price to pay for correctly functioning programs!\nNot unlocking a mutex due to an early return during an error condition\nResource leak (not calling pthread_mutex_destroy)\nUsing an uninitialized mutex or using a mutex that has already been destroyed\nLocking a mutex twice on a thread without unlocking first\nDeadlock\nMutex Implementation # So we have this cool data structure. How do we implement it? A naive, incorrect implementation is shown below. The unlock function simply unlocks the mutex and returns. The lock function first checks to see if the lock is already locked. If it is currently locked, it will keep checking again until another thread has unlocked the mutex. For the time being, we’ll avoid the condition that other threads are able to unlock a lock they don’t own and focus on the mutual exclusion aspect.\n// Version 1 (Incorrect!) void lock(mutex_t *m) { while(m-\u0026gt;locked) { /*Locked? Never-mind - loop and check again!*/ } m-\u0026gt;locked = 1; } void unlock(mutex_t *m) { m-\u0026gt;locked = 0; } Version 1 uses ‘busy-waiting’ unnecessarily wasting CPU resources. However, there is a more serious problem. We have a race-condition! If two threads both called lock concurrently, it is possible that both threads would read m_locked as zero. Thus both threads would believe they have exclusive access to the lock and both threads will continue.\nWe might attempt to reduce the CPU overhead a little by calling pthread_yield() inside the loop - pthread_yield suggests to the operating system that the thread does not use the CPU for a short while, so the CPU may be assigned to threads that are waiting to run. This still leaves the race-condition. We need a better implementation. We will talk about this later in the critical section part of this chapter. For now, we will talk about semaphores.\nAdvanced: Implementing a Mutex with hardware # We can use C11 Atomics to do that perfectly! A complete solution is detailed here. This is a spinlock mutex, https://locklessinc.com/articles/mutex_cv_futex/ implementations can be found online.\nFirst the data structure and initialization code.\ntypedef struct mutex_{ // We need some variable to see if the lock is locked atomic_int_least8_t lock; // A mutex needs to keep track of its owner so // Another thread can\u0026#39;t unlock it pthread_t owner; } mutex; #define UNLOCKED 0 #define LOCKED 1 #define UNASSIGNED_OWNER 0 int mutex_init(mutex* mtx){ // Some simple error checking if(!mtx){ return 0; } // Not thread-safe the user has to take care of this atomic_init(\u0026amp;mtx-\u0026gt;lock, UNLOCKED); mtx-\u0026gt;owner = UNASSIGNED_OWNER; return 1; } This is the initialization code, nothing fancy here. We set the state of the mutex to unlocked and set the owner to locked.\nint mutex_lock(mutex* mtx){ int_least8_t zero = UNLOCKED; while(!atomic_compare_exchange_weak_explicit (\u0026amp;mtx-\u0026gt;lock, \u0026amp;zero, LOCKED, memory_order_seq_cst, memory_order_seq_cst)){ zero = UNLOCKED; sched_yield(); // Use system calls for scheduling speed } // We have the lock now mtx-\u0026gt;owner = pthread_self(); return 1; } What does this code do? It initializes a variable that we will keep as the unlocked state. https://en.wikipedia.org/wiki/Compare-and-swap is an instruction supported by most modern architectures (on x86 it’s lock cmpxchg). The pseudocode for this operation looks like this.\nint atomic_compare_exchange_pseudo(int* addr1, int* addr2, int val){ if(*addr1 == *addr2){ *addr1 = val; return 1; }else{ *addr2 = *addr1; return 0; } } Except it is all done atomically meaning in one uninterruptible operation. What does the weak part mean? Atomic instructions are prone to spurious failures meaning that there are two versions to these atomic functions a strong and a weak part, strong guarantees the success or failure while weak may fail even when the operation succeeds. These are the same spurious failures that you’ll see in condition variables below. We are using weak because weak is faster, and we are in a loop! That means we are okay if it fails a little bit more often because we will keep spinning around anyway.\nInside the while loop, we have failed to grab the lock! We reset zero to unlocked and sleep for a little while. When we wake up we try to grab the lock again. Once we successfully swap, we are in the critical section! We set the mutex’s owner to the current thread for the unlock method and return successfully.\nHow does this guarantee mutual exclusion? When working with atomics we are unsure! But in this simple example, we can because the thread that can successfully expect the lock to be UNLOCKED (0) and swap it to a LOCKED (1) state is considered the winner. How do we implement unlock?\nint mutex_unlock(mutex* mtx){ if(unlikely(pthread_self() != mtx-\u0026gt;owner)){ return 0; // Can\u0026#39;t unlock a mutex if the thread isn\u0026#39;t the owner } int_least8_t one = 1; //Critical section ends after this atomic mtx-\u0026gt;owner = UNASSIGNED_OWNER; if(!atomic_compare_exchange_strong_explicit( \u0026amp;mtx-\u0026gt;lock, \u0026amp;one, UNLOCKED, memory_order_seq_cst, memory_order_seq_cst)){ //The mutex was never locked in the first place return 0; } return 1; } To satisfy the API, a thread can’t unlock the mutex unless the thread is the one who owns it. Then we unassign the mutex owner, because critical section is over after the atomic. We want a strong exchange because we don’t want to block. We expect the mutex to be locked, and we swap it to unlock. If the swap was successful, we unlocked the mutex. If the swap wasn’t, that means that the mutex was UNLOCKED and we tried to switch it from UNLOCKED to UNLOCKED, preserving the behavior of unlock.\nWhat is this memory order business? We were talking about memory fences earlier, here it is! We won’t go into detail because it is outside the scope of this course but in the scope of https://gcc.gnu.org/wiki/Atomic/GCCMM/AtomicSync. We need consistency to make sure no loads or stores are ordered before or after. A program need to create dependency chains for more efficient ordering.\n"},{"id":9,"href":"/cs341/en/chap4/","title":"Processes","section":"Preface","content":" Processes # Who needs process isolation? - Intel Marketing on Meltdown and Spectre To understand what a process is, you need to understand what an operating system is. An operating system is a program that provides an interface between hardware and user software as well as providing a set of tools that the software can use. The operating system manages hardware and gives user programs a uniform way of interacting with hardware as long as the operating system can be installed on that hardware. Although this idea sounds like it is the end-all, we know that there are many different operating systems with their own quirks and standards. As a solution to that, there is another layer of abstraction: POSIX or portable operating systems interface. This is a standard (or many standards now) that an operating system must implement to be POSIX compatible – most systems that we’ll be studying are almost POSIX compatible due more to political reasons.\nBefore we talk about POSIX systems, we should understand what the idea of a kernel is generally. In an operating system (OS), there are two spaces: kernel space and user space. Kernel space is a power operating mode that allows the system to interact with the hardware and has the potential to destroy your machine. User space is where most applications run because they don’t need this level of power for every operation. When a user space program needs additional power, it interacts with the hardware through a system call that is conducted by the kernel. This adds a layer of security so that normal user programs can’t destroy your entire operating system. For the purposes of our class, we’ll talk about single machine multiple user operating systems. This is where there is a central clock on a standard laptop or desktop. Other OSes relax the central clock requirement (distributed) or the “standardness” of the hardware (embedded systems). Other invariants make sure events happen at particular times too.\nThe operating system is made up of many different pieces. There may be a program running to handle incoming USB connections, another one to stay connected to the network, etc. The most important one is the kernel – although it might be a set of processes – which is the heart of the operating system. The kernel has many important tasks. The first of which is booting.\nThe computer hardware executes code from read-only memory, called firmware.\nThe firmware executes a bootloader, which often conforms to the Extensible Firmware Interface (EFI), which is an interface between the system firmware and the operating system.\nThe bootloader’s boot manager loads the operating system kernels, based on the boot settings.\nYour kernel executes init to https://en.wikipedia.org/wiki/Bootstrapping itself from nothing.\nThe kernel executes startup scripts like starting networking and USB handling.\nThe kernel executes userland scripts like starting a desktop, and you get to use your computer!\nWhen a program is executing in user space, the kernel provides some important services to programs in User space.\nScheduling processes and threads\nHandling synchronization primitives (futexes, mutexes, semaphores, etc.)\nProviding system calls such as write or read\nManaging virtual memory and low-level binary devices such as USB drivers\nManaging filesystems\nHandling communication over networks\nHandling communication between processes\nDynamically linking libraries\nThe list goes on and on.\nThe kernel creates the first process init.d (an alternative is system.d). init.d boots up programs such as graphical user interfaces, terminals, etc – by default, this is the only process explicitly created by the system. All other processes are instantiated by using the system calls fork and exec from that single process.\n"},{"id":10,"href":"/cs341/en/chap7/Semaphore/","title":"Semaphore","section":"Synchronization","content":" Semaphore # A semaphore is another synchronization primitive. It is initialized to some value. Threads can either sem_wait or sem_post which lowers or increases the value. If the value reaches zero and a wait is called, the thread will be blocked until a post is called.\nUsing a semaphore is as easy as using a mutex. First, decide if on the initial value, for example the number of remaining spaces in an array. Unlike pthread mutex there are no shortcuts to creating a semaphore - use sem_init.\n#include \u0026lt;semaphore.h\u0026gt; sem_t s; int main() { sem_init(\u0026amp;s, 0, 10); // returns -1 (=FAILED) on OS X sem_wait(\u0026amp;s); // Could do this 10 times without blocking sem_post(\u0026amp;s); // Announce that we\u0026#39;ve finished (and one more resource item is available; increment count) sem_destroy(\u0026amp;s); // release resources of the semaphore } When using a semaphore, wait and post can be called from different threads! Unlike a mutex, the increment and decrement can be from different threads.\nThis becomes especially useful if you want to use a semaphore to implement a mutex. A mutex is a semaphore that always waits before it posts. Some textbooks will refer to a mutex as a binary semaphore. You do have to be careful to never add more than one to a semaphore or otherwise your mutex abstraction breaks. That is usually why a mutex is used to implement a semaphore and vice versa.\nInitialize the semaphore with a count of one.\nReplace pthread_mutex_lock with sem_wait\nReplace pthread_mutex_unlock with sem_post\nsem_t s; sem_init(\u0026amp;s, 0, 1); sem_wait(\u0026amp;s); // Critical Section sem_post(\u0026amp;s); But be warned, it isn’t the same! A mutex can handle what we call lock inversion well. Meaning the following code breaks with a traditional mutex, but produces a race condition with threads.\n// Thread 1 sem_wait(\u0026amp;s); // Critical Section sem_post(\u0026amp;s); // Thread 2 // Some threads want to see the world burn sem_post(\u0026amp;s); // Thread 3 sem_wait(\u0026amp;s); // Not thread-safe! sem_post(\u0026amp;s); If we replace it with mutex lock, it won’t work now.\n// Thread 1 mutex_lock(\u0026amp;s); // Critical Section mutex_unlock(\u0026amp;s); // Thread 2 // Foiled! mutex_unlock(\u0026amp;s); // Thread 3 mutex_lock(\u0026amp;s); // Now it\u0026#39;s thread-safe mutex_unlock(\u0026amp;s); Also, binary semaphores are different than mutexes because one thread can unlock a mutex from a different thread.\nSignal Safety # Also, sem_post is one of a handful of functions that can be correctly used inside a signal handler pthread_mutex_unlock is not. We can release a waiting thread that can now make all of the calls that we disallowed to call inside the signal handler itself e.g. printf. Here is some code that utilizes this;\n#include \u0026lt;stdio.h\u0026gt; #include \u0026lt;pthread.h\u0026gt; #include \u0026lt;signal.h\u0026gt; #include \u0026lt;semaphore.h\u0026gt; #include \u0026lt;unistd.h\u0026gt; sem_t s; void handler(int signal) { sem_post(\u0026amp;s); /* Release the Kraken! */ } void *singsong(void *param) { sem_wait(\u0026amp;s); printf(\u0026#34;Waiting until a signal releases...\\n\u0026#34;); } int main() { int ok = sem_init(\u0026amp;s, 0, 0 /* Initial value of zero*/); if (ok == -1) { perror(\u0026#34;Could not create unnamed semaphore\u0026#34;); return 1; } signal(SIGINT, handler); // Too simple! See Signals chapter pthread_t tid; pthread_create(\u0026amp;tid, NULL, singsong, NULL); pthread_exit(NULL); /* Process will exit when there are no more threads */ } Other uses for semaphores are keeping track of empty spaces in arrays. We will discuss these in the thread-safe data structures section.\n"},{"id":11,"href":"/cs341/en/chap7/","title":"Synchronization","section":"Preface","content":" Synchronization # When multithreading gets interesting "},{"id":12,"href":"/cs341/en/chap2/Systems_Architecture/","title":"Systems Architecture","section":"Background","content":" Systems Architecture # This section is a short review of System Architecture topics that you’ll need for System Programming.\nAssembly # What is assembly? Assembly is the lowest that you’ll get to machine language without writing 1’s and 0’s. Each computer has an architecture, and that architecture has an associated assembly language. Each assembly command has a 1:1 mapping to a set of 1’s and 0’s that tell the computer exactly what to do. For example, the following in the widely used x86 Assembly language add one to the memory address 20 (Wikibooks #ref-wiki:xxx) – you can also look in (Guide #ref-guide2011intel) Section 2A under the add instruction though it is more verbose.\nadd BYTE PTR [0x20], 1 Why do we mention this? Because it is important that although you are going to be doing most of this class in C, that this is what the code is translated into. Serious implications arise for race conditions and atomic operations. Atomic Operations\nAn operation is atomic if no other processor should interrupt it. Take for example the above assembly code to add one to a register. In the architecture, it may actually have a few different steps on the circuit. The operation may start by fetching the value of the memory from the stick of ram, then storing it in the cache or a register, and then finally writing back (Schweizer, Besta, and Hoefler #ref-schweizer2015evaluating) – under the description for fetch-and-add though your micro-architecture may vary. Or depending on performance operations, it may keep that value in cache or in a register which is local to that process – try dumping the -O2 optimized assembly of incrementing a variable. The problem comes in if two processors try to do it at the same time. The two processors could at the same time copy the value of the memory address, add one, and store the same result back, resulting in the value only being incremented once. That is why we have a special set of instructions on modern systems called atomic operations. If an instruction is atomic, it makes sure that only one processor or thread performs any intermediate step at a time. With x86 this is done by the lock prefix (Guide #ref-guide2011intel, 1120).\nlock add BYTE PTR [0x20], 1 Why don’t we do this for everything? It makes commands slower! If every time a computer does something it has to make sure that the other cores or processors aren’t doing anything, it’ll be much slower. Most of the time we differentiate these with special consideration. Meaning, we will tell you when we use something like this. Most of the time you can assume the instructions are unlocked. Caching\nAh yes, Caching. One of computer science’s greatest problems. Caching that we are referring is processor caching. If a particular address is already in the cache when reading or writing, the processor will perform the operation on the cache such as adding and update the actual memory later because updating memory is slow (Intel #ref-intel2015improving Section 3.4). If it isn’t, the processor requests a chunk of memory from the memory chip and stores it in the cache, kicking out the least recently used page – this depends on caching policy, but Intel’s does use this. This is done because the l3 processor cache is roughly three times faster to reach than the memory in terms of time (Levinthal #ref-levinthal2009performance, 22) though exact speeds will vary based on the clock speed and architecture. Naturally, this leads to problems because there are two different copies of the same value, in the cited paper this refers to an unshared line. This isn’t a class about caching, know how this could impact your code. A short but non-complete list could be\nRace Conditions! If a value is stored in two different processor caches, then that value should be accessed by a single thread.\nSpeed. With a cache, your program may look faster mysteriously. Just assume that reads and writes that either happened recently or are next to each other in memory are fast.\nSide effects. Every read or write effects the cache state. While most of the time this doesn’t help or hurt, it is important to know. Check the Intel programmer guide on the lock prefix for more information.\nInterrupts # Interrupts are a important part of system programming. An interrupt is internally an electrical signal that is delivered to the processor when something happens – this is a hardware interrupt (“Chapter 3. Hardware Interrupts,” #ref-redhat_hardware_int). Then the hardware decides if this is something that it should handle (i.e. handling keyboard or mouse input for older keyboard and mouses) or it should pass to the operating system. The operating system then decides if this is something that it should handle (i.e. paging a memory table from disk) or something the application should handle (i.e. a SEGFAULT). If the operating system decides that this is something that the process or program should take care of, it sends a software fault and that software fault is then propagated. The application then decides if it is an error (SEGFAULT) or not (SIGPIPE for example) and reports to the user. Applications can also send signals to the kernel and to the hardware as well. This is an oversimplification because there are certain hardware faults that can’t be ignored or masked away, but this class isn’t about teaching you to build an operating system.\nAn important application of this is this is how system calls are served! There is a well-established set of registers that the arguments go in according to the kernel as well as a system call “number” again defined by the kernel. Then the operating system triggers an interrupt which the kernel catches and serves the system call (Garg #ref-garg_2006).\nOperating system developers and instruction set developers alike didn’t like the overhead of causing an interrupt on a system call. Now, systems use SYSENTER and SYSEXIT which has a cleaner way of transferring control safely to the kernel and safely back. What safely means is obvious out of the scope for this class, but it persists.\nOptional: Hyperthreading # Hyperthreading is a new technology and is in no way shape or form multithreading. Hyperthreading allows one physical core to appear as many virtual cores to the operating system (Guide #ref-guide2011intel, P.51). The operating system can then schedule processes on these virtual cores and one core will execute them. Each core interleaves processes or threads. While the core is waiting for one memory access to complete, it may perform a few instructions of another process thread. The overall result is more instructions executed in a shorter time. This potentially means that you can divide the number of cores you need to power smaller devices.\nThere be dragons here though. With hyperthreading, you must be wary of optimizations. A famous hyperthreading bug that caused programs to crash if at least two processes were scheduled on a physical core, using specific registers, in a tight loop. The actual problem is better explained through an architecture lens. But, the actual application was found through systems programmers working on OCaml’s mainline (Leroy #ref-leroy_2017).\n"},{"id":13,"href":"/cs341/en/chap3/","title":"The C Programming Language","section":"Preface","content":" The C Programming Language # If you want to teach systems, don’t drum up the programmers, sort the issues, and make PRs. Instead, teach them to yearn for the vast and endless C. - Antoine de Saint-Exupéry (With edits) Note: This chapter is long and goes into a lot of detail. Feel free to gloss over parts with which you have experience in.\nC is the de-facto programming language to do serious system serious programming. Why? Most kernels have their API accessible through C. The Linux kernel1 and the XNU kernel2 of which MacOS is based on are written in C and have C API - Application Programming Interface. The Windows Kernel uses C++, but doing system programming on that is much harder on windows that UNIX for novice system programmers. C doesn’t have abstractions like classes and Resource Acquisition Is Initialization (RAII) to clean up memory. C also gives you much more of an opportunity to shoot yourself in the foot, but it lets you do things at a much more fine-grained level.\nLove, Robert. 2010. Linux Kernel Development. 3rd ed. Addison-Wesley Professional.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nInc., Apple. 2017. “XNU Kernel.” GitHub Repository. https://github.com/apple/darwin-xnu; GitHub\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n"},{"id":14,"href":"/cs341/en/chap6/","title":"Threads","section":"Preface","content":" Threads # If you think your programs crashing before, wait until they crash ten times as fast A thread is short for ‘thread-of-execution’. It represents the sequence of instructions that the CPU has and will execute. To remember how to return from function calls, and to store the values of automatic variables and parameters a thread uses a stack. Almost weirdly, a thread is a process, meaning that creating a thread is similar to fork, except there is no copying meaning no copy on write. What this allows is for a process to share the same address space, variables, heap, file descriptors and etc. The actual system call to create a thread is similar to fork. It’s clone. We won’t go into the specifics, but you can read the http://man7.org/linux/man-pages/man2/clone.2.html keeping in mind that it is outside the direct scope of this course. LWP or Lightweight Processes or threads are preferred to forking for a lot of scenarios because there is a lot less overhead creating them. But in some cases, notably python uses this, multiprocessing is the way to make your code faster.\n"},{"id":15,"href":"/cs341/en/chap2/Valgrind/","title":"Valgrind","section":"Background","content":" Valgrind # Valgrind is a suite of tools designed to provide debugging and profiling tools to make your programs more correct and detect some runtime issues (“4. Memcheck: A Memory Error Detector,” #ref-valgrind). The most used of these tools is Memcheck, which can detect many memory-related errors that are common in C and C++ programs and that can lead to crashes and unpredictable behavior (for example, unfreed memory buffers). To run Valgrind on your program:\nvalgrind --leak-check=full --show-leak-kinds=all myprogram arg1 arg2 Arguments are optional and the default tool that will run is Memcheck. The output will be presented in the form: the number of allocations, frees, and errors. Suppose we have a simple program like this:\n#include \u0026lt;stdlib.h\u0026gt; void dummy_function() { int* x = malloc(10 * sizeof(int)); x[10] = 0; // error 1: Out of bounds write, as you can see here we write to an out of bound memory address. } // error 2: Memory Leak, x is allocated at function exit. int main(void) { dummy_function(); return 0; } This program compiles and runs with no errors. Let’s see what Valgrind will output.\n==29515== Memcheck, a memory error detector ==29515== Copyright (C) 2002-2015, and GNU GPL\u0026#39;d, by Julian Seward et al. ==29515== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==29515== Command: ./a ==29515== ==29515== Invalid write of size 4 ==29515== at 0x400544: dummy_function (in /home/rafi/projects/exocpp/a) ==29515== by 0x40055A: main (in /home/rafi/projects/exocpp/a) ==29515== Address 0x5203068 is 0 bytes after a block of size 40 alloc\u0026#39;d ==29515== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==29515== by 0x400537: dummy_function (in /home/rafi/projects/exocpp/a) ==29515== by 0x40055A: main (in /home/rafi/projects/exocpp/a) ==29515== ==29515== ==29515== HEAP SUMMARY: ==29515== in use at exit: 40 bytes in 1 blocks ==29515== total heap usage: 1 allocs, 0 frees, 40 bytes allocated ==29515== ==29515== LEAK SUMMARY: ==29515== definitely lost: 40 bytes in 1 blocks ==29515== indirectly lost: 0 bytes in 0 blocks ==29515== possibly lost: 0 bytes in 0 blocks ==29515== still reachable: 0 bytes in 0 blocks ==29515== suppressed: 0 bytes in 0 blocks ==29515== Rerun with --leak-check=full to see details of leaked memory ==29515== ==29515== For counts of detected and suppressed errors, rerun with: -v ==29515== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) Invalid write: It detected our heap block overrun, writing outside of an allocated block.\nDefinitely lost: Memory leak — you probably forgot to free a memory block.\nValgrind is a effective tool to check for errors at runtime. C is special when it comes to such behavior, so after compiling your program you can use Valgrind to fix errors that your compiler may miss and that usually happens when your program is running.\nFor more information, you can refer to the manual1\n“4. Memcheck: A Memory Error Detector.” n.d. Valgrind. http://valgrind.org/docs/manual/mc-manual.html.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n"}] |