{"id":315,"date":"2012-04-16T07:29:01","date_gmt":"2012-04-16T07:29:01","guid":{"rendered":"http:\/\/www.plugged.in\/?p=315"},"modified":"2012-04-16T07:29:01","modified_gmt":"2012-04-16T07:29:01","slug":"find-the-largest-files","status":"publish","type":"post","link":"https:\/\/www.veriteknik.net.tr\/en\/find-the-largest-files\/","title":{"rendered":"Find the Largest Files"},"content":{"rendered":"

The other day I was working on a server and needed the LARGEST files on some directory – including its subdirectories.<\/p>\n

As it turns out, it’s a very simple task limiting the file size you want with the output of the find<\/strong> tool.<\/p>\n

The -size<\/strong> argument will define the borders of your output. Let’s say you want to find the files smaller than 50 MB on your server,<\/p>\n

find \/ -type f -size -50M<\/pre>\n
Well, this will print out the full path and the file names, you won’t know which file is at what size. So to improve this, we can execute an ls<\/strong> command on each output,<\/p>\n
find \/ -type f -size -50M -exec ls -lh {} \\;<\/pre>\n
On this command, the {}<\/strong> refers to the output of each find command, and the \\;<\/strong> is mandatory since we need to tell find<\/strong> where our -exec<\/strong> line ends, hence the escape character. (\\<\/strong>)<\/p>\n
Even though this looks good, we can keep improving by printing out the file size all in the same units. (Let’s say, megabytes) The problem with that is, the ls<\/strong> command can printout with the specified block size limit but it will take that block size quantised<\/a><\/em>, meaning the output will only be the exact multiplicands of that block size. So if our block size is set to 1 MB and a file is 900 KB, ls<\/strong> will output it as 1 MB.<\/p>\n
Although this is not so accurate, we can always work it around using awk<\/strong> to calculate the numeric values for us. Since ls<\/strong> normally prints out the file size in bytes<\/em>, we can divide them to become actual megabytes<\/em>. The line below will printout ls<\/strong> with actual megabytes.<\/p>\n
ls -l | awk '{print $1 \" \" $2 \" \" $3 \" \" $4 \" \" $5\/1048576 \" \" $6 \" \" $7 \" \" $8 \" \" $9}'<\/pre>\n
Well, we only need the 5th<\/em> column and the 9th<\/em> column which are the size<\/em> and the path<\/em> respectively, so the command below will suffice :<\/p>\n
ls -l | awk '{ print $5\/1048576 \" \" $9 }'<\/pre>\n
As you can see, we had to use a pipe<\/em> to get things done here. So we need to use this pipe in our exec<\/em> part of our find<\/strong> command, which is another problem. Well, the work around for this is to -exec<\/strong> a shell instance<\/em> and pass the whole ls<\/strong> and awk<\/strong> line including the pipes so that new shell instance will handle things for us.<\/p>\n
find \/ -type f -size -50M -size +20M -exec sh -c \"ls -l '{}'|awk '{print \\$5\/1048576 \" MB: \" \\$9}'\" \\;<\/pre>\n
Ok, let’s have a look at the command above. As you can see we narrowed our limits further, by getting only the files smaller than 50 MB and larger than 20 MB. We also passed our whole command with a shell instance. On this instance, the argument for ls<\/strong> was passed with the {}<\/strong> method. We apostrophized it (”<\/strong>) due to the possibility of having spaces in the filename, which would have caused a problem. After that, we’ve piped our output to awk<\/strong>, divided the bytes<\/em>, and added a string ” MB: ” right before printing the 9th<\/em> column which is the file path. Don’t forget that we should escape the $5<\/em> and $9<\/em> using the escape character \\<\/strong> since we don’t want the whole find<\/strong> line to process it before our awk<\/strong> does.<\/p>\n
Well, the good thing is we have necessary output, the bad thing is that, it isn’t in order! So let’s make things even prettier and sort<\/em> them, while making the output of each “MB:<\/strong>” bold to get some eye candy.<\/p>\n
find \/ -type f -size -50M -size +20M -exec sh -c \"ls -l '{}'|awk '{print \\$5\/1048576 \\\" \\033[1mMB:\\033[0;0m \\\" \\$9}'\" \\; | sort -nr -k1<\/pre>\n
As you can see here, we’ve piped the find<\/strong> command to the sort<\/strong>, not the shell<\/strong> instance that we invoked in the find<\/strong> command, that’s why the pipe is right after our \\;<\/strong> character.<\/p>\n
On this command, find will also search inside \/proc<\/strong> directory, which is a living directory, so during the search some files will be created and destroyed rapidly, which will cause some annoying outputs saying “file not found”. To avoid that, let’s tell find NOT<\/strong> to search the \/proc<\/strong> directory using the -prune<\/strong> argument.<\/p>\n
find \/ -path '\/proc' -prune -o -type f -size -50M -size +20M -exec sh -c \"ls -l '{}'|awk '{print \\$5\/1048576 \\\" \\033[1mMB:\\033[0;0m \\\" \\$9}'\" \\; | sort -nr -k1<\/pre>\n
You can add new directories to prune with the -path ‘\/new\/directory\/to\/prune’ -prune -o<\/strong> method.<\/p>\n
Hope this helps.<\/p>\n","protected":false},"excerpt":{"rendered":"
The other day I was working on a server and needed the LARGEST files on some directory – including its subdirectories. As it turns out, it’s a very simple task limiting the file size you want with the output of the find tool. The -size argument will define the borders of your output. Let’s say […]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","footnotes":""},"categories":[370,372],"tags":[471,366,469,472,473,474,371,470,475,476],"yst_prominent_words":[1483,1481,1477,1482,1486,1474,336,1261,1480,605,1475,711,705,1473,1485,1478,1484,1476,226,1479],"class_list":["post-315","post","type-post","status-publish","format-standard","hentry","category-linux","category-linux_help","tag-awk","tag-centos","tag-console","tag-file","tag-find","tag-large","tag-linux-2","tag-ssh","tag-terminal","tag-unix"],"jetpack_featured_media_url":"","uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"Mustafa Emre Ayd\u0131n","author_link":"https:\/\/www.veriteknik.net.tr\/en\/author\/eaydin\/"},"uagb_comment_info":0,"uagb_excerpt":"The other day I was working on a server and needed the LARGEST files on some directory – including its subdirectories. As it turns out, it’s a very simple task limiting the file size you want with the output of the find tool. The -size argument will define the borders of your output. Let’s say…","_links":{"self":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts\/315","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/comments?post=315"}],"version-history":[{"count":1,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts\/315\/revisions"}],"predecessor-version":[{"id":7296,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/posts\/315\/revisions\/7296"}],"wp:attachment":[{"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/media?parent=315"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/categories?post=315"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/tags?post=315"},{"taxonomy":"yst_prominent_words","embeddable":true,"href":"https:\/\/www.veriteknik.net.tr\/en\/wp-json\/wp\/v2\/yst_prominent_words?post=315"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}