How to ensure data-driven export strategies

List of the contents of this article:

• 1,How to ensure data-driven export strategies Exploration of the scheme of automatically cleaning up a large number of files under Linux
• 2, The characteristics of the distributed file system and the parallel file system
• 3, Why the file system is introduced

Exploration of the scheme of automatically cleaning up a large number of files under Linux

1. This is the basic idea of automatic file cleaning, generating a list of files to be deleted according to preset conditions, and then performing regular cleaning tasks to perform deletion operations.For expired files, their common symbol is the timestamp. Depending on different file systems, there may be different time attributes such as file creation time, access time, expiration time, etc.

2. Just get rid of a directory containing tens of thousands of files (/var/log/samba), try it. Note: You can also use rm -fr (path + file to be deleted).

3. Empty the search bar and now search for "linux-headers". If they have been installed, you should get a list of all the installed Linux title files related to the kernel installed on the system. The version number is accompanied by the kernel, so be sure that you have not cleared the title file of the current version.

4,BleachBit BleachBit is a convenient tool that can protect user privacy while cleaning up disk space, and works on both Linux and Windows system platforms.

Characteristics of distributed file system and parallel file system

1. Simply put, non-local direct connection and network connection is a distributed file system; in the distributed file system, service The device node is composed of multiple, which is a cluster file system; which supports parallel applications (such as MPI), which is a parallel file system.

2. The advantage of distributed systems is networking, which provides the possibility for the expansion of the system. However,The biggest problem of distribution is also various uncertainties brought about by networking. From an industrial perspective, the network problem is the original sin of distributed systems, which brings great difficulties and challenges to the design and implementation of distributed systems.

3. Low-latency data. For example, applications that interact with users require data to be responded to within milliseconds or seconds. Because Hadoop is optimized for high data throughput and sacrifices the delay in data acquisition, it is not suitable for low latency with Hadoop.

4. Distributed refers to a system in which multiple systems work together to complete a specific task. It is that different systems are deployed on different servers, and the servers call each other. It's like many people doing different things together. Distributed is to solve the problem of centralized management. It is too slow to superimpose all tasks on one node for processing.

5. Distributed Software Systems is a software system that supports distributed processing. It is a system that performs tasks on a multi-processor architecture interconnected by a communication network.

Why the file system is introduced

[Answer]: When users request to perform multiple read/write or other operations on a file, they have to start from the search directory every time, which is a waste of time and inefficient. In order to avoid repeatedly retrieving the directory, the file system call "Open" has been introduced in most 0S.

From the perspective of the system, the file system is a system that organizes and allocates the space of file storage devices, is responsible for file storage and protects and retrieves stored files.Specifically, it is responsible for establishing files for users, depositing, reading, modifying and dumping files, controlling file access, and undoing files when users no longer use them.

From the user's perspective, the main purpose of introducing a file system is to achieve name-by-name access to files.

There are many types of file systems, and the operating system hopes to provide a unified interface for users, so the intermediate layer is introduced in the user layer and the file system layer, which is called Virtual File System (VFS) .

File management File is an important concept in the operating system. It is a collection of information stored on the computer with the computer hard disk as the carrier.

The file system consists of three partsComposition: software related to file management, managed files and data structures required to implement file management. From the perspective of the system, the file system is a system that organizes and allocates file storage space, is responsible for the storage of files, and protects and retrieves stored files.