Hadoop HDFS Command Cheatsheet List Files hdfs dfs -ls / List all the files/directories for the given hdfs destination path. Directories are listed as plain files. In this case, this command will list the details of hadoop folder. Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864). Recursively list all files in hadoop directory and all subdirectories in hadoop directory. hdfs dfs -ls -d /hadoop hdfs dfs -ls -h /data hdfs dfs -ls -R /hadoop List all the files matching the pattern. In this case, it will list all the files inside hadoop directory which starts with 'dat'. hdfs dfs -ls /hadoop/dat* Read/Write Files HDFS Command that takes a source file and outputs the file in text format on the terminal. The allowed formats are zip and TextRecordInputStream. This command will display the content of the HDFS file test on your stdout . hdfs dfs -text /hadoop/derby.log hdfs dfs -cat /hadoop/test hdfs dfs -appendToFile /home/ubuntu/test1 /hadoop/text2 Appends the content of a local file test1 to a hdfs file test2. Upload/Download Files hdfs dfs -put /home/ubuntu/sample /hadoop Copies the file from local file system to HDFS. Copies the file from local file system to HDFS, and in case the local already exits in the given destination path, using -f option with put command will overwrite it. hdfs dfs -put -f /home/ubuntu/sample /hadoop hdfs dfs -put -l /home/ubuntu/sample /hadoop Copies the file from local file system to HDFS. Allow DataNode to lazily persist the file to disk. Forces replication factor of 1. hdfs dfs -put -p /home/ubuntu/sample /hadoop Copies the file from local file system to HDFS. Passing -p preserves access and modification times, ownership and the mode. hdfs dfs -get /newfile /home/ubuntu/ Copies the file from HDFS to local file system. hdfs dfs -get -p /newfile /home/ubuntu/ Copies the file from HDFS to local file system. Passing -p preserves access and modification times, ownership and the mode. hdfs dfs -get /h

阅读全文