VTU Notes | 18EC72 | BIG DATA ANALYTICS

VTU Module-2 | Hadoop Distributed File System Basics

Module-2

Hadoop Distributed File System Basics (T2): A Quick Guide

Highly Scalable: Stores data across numerous commodity hardware nodes, increasing capacity and processing power.
Fault Tolerant: Replicates data across nodes, ensuring availability even if some nodes fail.
High Throughput: Designed for fast read and write operations for large datasets.
Simple File System Model: Provides a familiar hierarchical structure for organizing data.
Stream Access: Optimized for reading large files sequentially rather than random access.

NameNode: The central authority, managing the file system namespace and block locations.
DataNode: Stores data blocks and communicates with the NameNode for instructions.
Secondary NameNode: Periodically backs up the NameNode metadata for disaster recovery.
Client: An application or user interacting with the HDFS cluster.

hdfs dfs -ls: Lists directories and files in HDFS.
hdfs dfs -put: Uploads files from local machine to HDFS.
hdfs dfs -get: Downloads files from HDFS to local machine.
hdfs dfs -mkdir: Creates new directories in HDFS.
hdfs dfs -rm: Deletes files or directories in HDFS.
hdfs dfs -copyFromLocal: Copies files from local machine to HDFS with progress reporting.
hdfs dfs -copyToLocal: Copies files from HDFS to local machine with progress reporting.

HDFS is designed for large data processing, not small files or frequent modification.
Security features are available but need to be configured separately.
HDFS works best with MapReduce and YARN for distributed processing.

This summary gives a basic understanding of HDFS design, components, and user commands. Feel free to ask further questions for deeper exploration!

Can we download the notes?

Yes, you can download the notes by going to the Module Topics and clicking on the View/Download Module Notes.
How often notes are updated on AcquireHowTo?

We try our best to provide update notes to our users, so we keep updating them once a week.
Do you provide only one specific university note?

No, Our team tries to work hard to provide notes from multiple universities like VTU, IP, DTU, Amity, etc, and from multiple courses like B.E, B.Tech, BBA, MBA, BCA, etc.
Do the Notes you provide belongs to you?

No, the notes we provide belong to the only creator of that notes. May some note belongs to us but not all. AcquireHowTo is a notes providing platform that provide notes from different sources at one place to help the students.