{"id":422,"date":"2012-05-22T12:12:21","date_gmt":"2012-05-22T12:12:21","guid":{"rendered":"http:\/\/www.plugged.in\/?p=422"},"modified":"2012-05-22T12:12:21","modified_gmt":"2012-05-22T12:12:21","slug":"system-control-over-python-with-pipes","status":"publish","type":"post","link":"https:\/\/www.veriteknik.net.tr\/en\/system-control-over-python-with-pipes\/","title":{"rendered":"System Control Over Python With Pipes"},"content":{"rendered":"<p>Even though major system scripting is usually done via bash (or other shell) scripting, it&#8217;s almost as common to see Python as a system administration scripting platform. Today, most major installers, daemons and package management software are written in Python.<\/p>\n<p>While using Python as a system scripting tool, it&#8217;s essential to keep using the standard gnu tools if possible, since these tools are reliable, fast and have been tested for years by thousands of other system administrators as well.<\/p>\n<p>The best way to keep using your good old fashioned gnu tools via Python is opening pipes to such tools. Even though it is possible to use the <strong>os<\/strong> module to do such tasks, the <strong>subprocess<\/strong> module is more reliable and you have more control over the tasks.<\/p>\n<p>Below you&#8217;ll see two different methods to approach the &#8220;echo&#8221; tool via Python.<\/p>\n<pre class=\"brush: python; gutter: true; first-line: 1\">&gt;&gt;&gt; import os, subprocess\n&gt;&gt;&gt; os.system(\"echo bla\")\nbla\n0\n&gt;&gt;&gt; subprocess.Popen([\"echo\",\"bla\"],stdout=subprocess.PIPE).communicate()[0]\n'bla\\n'<\/pre>\n<p>As you can see, both give us more or less the same output, but the subprocess module gives us a lot more control. First of all it has pipe support, universal newline support, better handling of exceptions etc.<\/p>\n<p>Below is a method I use to determine the number of processor on the system. Using this, you can find out if you have multiple cores on the system.<\/p>\n<pre class=\"brush: python; gutter: true; first-line: 1\">cpu_num=subprocess.Popen([\"grep\",\"-c\",\"processor\",\"\/proc\/cpuinfo\"],stdout=subprocess.PIPE).communicate()[0]<\/pre>\n<p>As you can see, what is does is very very simple, it just counts the number of word &#8220;processor&#8221; in the <strong>\/proc\/cpuinfo<\/strong> file using grep.<\/p>\n<p>Don&#8217;t forget that it is very easy to pipe commands through each other with popen, here&#8217;s an example.<\/p>\n<pre class=\"brush: python; gutter: true; first-line: 1\">&gt;&gt;&gt; import subprocess\n&gt;&gt;&gt; p1=subprocess.Popen([\"cat\",\"\/proc\/cpuinfo\"],stdout=subprocess.PIPE)\n&gt;&gt;&gt; p2=subprocess.Popen([\"grep\",\"-B\",\"2\",\"-A\",\"2\",\"processor\"],stdin=p1.stdout,stdout=subprocess.PIPE)\n&gt;&gt;&gt; output = p2.communicate()[0]\n&gt;&gt;&gt; print output\nprocessor : 0\nvendor_id : GenuineIntel\ncpu family : 15<\/pre>\n<p>Using the popen, it is very easy to get information on processes via the pgrep and ps tools. Below is a script when run with the valid parameters, checks if any of it&#8217;s processes uses more (or equal) to the percentage of the CPU specified, if true, kills the process and tries to send a message to the affiliated terminal window.<\/p>\n<p>To accomplish this, we get the pid&#8217;s of the processes via pgrep, send it to ps so to check the cpu percentage and which tty (or pts) it&#8217;s running on. After the check, if necessary, the PID is killed and a message is sent to the terminal it was running on.<\/p>\n<pre class=\"brush: python; gutter: true; first-line: 1\">#!\/usr\/bin\/python\nimport subprocess, sys, string\n\nif len(sys.argv) != 3 :\n    print \"Usage : cpukiller.py &lt;process-to-kill&gt; &lt;max-cpu-percentage&gt;\"\n    raise SystemExit\n\nproc_to_kill = sys.argv[1]\ncpu_to_kill = sys.argv[2]\n\nfor i in cpu_to_kill :\n    if i not in string.digits :\n        print \"max-cpu-percentage must consist of only digits.\"\n        raise SystemExit\n\ncpu_to_kill = int(cpu_to_kill)\n\ntry :\n\ta=subprocess.Popen([\"pgrep\",proc_to_kill],stdout=subprocess.PIPE).communicate()[0]\n\tif a == '' :\n\t\traise SystemExit\n\telse :\n\t\tprocc = subprocess.Popen([\"ps --no-headers -o pid,pcpu,tname -p $(pgrep %s)\"%proc_to_kill],shell=True,stdout=subprocess.PIPE).communicate()[0]\n\t\tprocc=procc.strip()\nexcept : raise SystemExit \nfor lines in procc.split('\\n') :\n    if lines != '' :\n        l=lines.split()\n        if int(l[1].split('.')[0]) &gt;= cpu_to_kill :\n            if l[2] != '?' :\n                try :\n                    terminal = open('\/dev\/'+l[2],'a')\n                    subprocess.Popen([\"echo\",\"You have exceeded your CPU limit, process with PID %s will be terminated\"%l[0]],stdout=terminal).communicate()[0]\n                except :\n                    print \"Couldn't tell the user.\"\n                    pass\n            try :\n                killer = subprocess.Popen([\"kill\",\"-9\",l[0]],stdout=subprocess.PIPE).communicate()[0]\n            except :\n                print \"Couldn't kill the process.\"\n                pass\n        else : pass<\/pre>\n<p>Try to use it, understand it, and expand it!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Even though major system scripting is usually done via bash (or other shell) scripting, it&#8217;s almost as common to see Python as a system administration scripting platform. Today, most major installers, daemons and package management software are written in Python. While using Python as a system scripting tool, it&#8217;s essential to keep using the standard [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","footnotes":""},"categories":[370,413,448,450,381],"tags":[513,514,515,449],"yst_prominent_words":[1151,1147,1140,1144,1154,1152,1145,1131,1138,1148,1143,713,1150,1141,1153,641,1139,1149,1146,1142],"class_list":["post-422","post","type-post","status-publish","format-standard","hentry","category-linux","category-linux-optimization","category-programm-in","category-python-programming","category-uncategorized","tag-administration","tag-pipe","tag-popen","tag-python-2"],"jetpack_featured_media_url":"","uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"Mustafa Emre Ayd\u0131n","author_link":"https:\/\/www.veriteknik.net.tr\/en\/author\/eaydin\/"},"uagb_comment_info":0,"uagb_excerpt":"Even though major system scripting is usually done via bash (or other shell) scripting, it&#8217;s almost as common to see Python as a system administration scripting platform. Today, most major installers, daemons and package management software are written in Python. While using Python as a system scripting tool, it&#8217;s essential to keep using the standard&hellip;","_links":{"self":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts\/422","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/comments?post=422"}],"version-history":[{"count":0,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts\/422\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/media?parent=422"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/categories?post=422"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/tags?post=422"},{"taxonomy":"yst_prominent_words","embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/yst_prominent_words?post=422"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}