php删除文本文件中重复行的方法

本文实例讲述了php删除文本文件中重复行的方法。分享给大家供大家参考。具体分析如下：

本文实例讲述了php删除文本文件中重复行的方法。分享给大家供大家参考。具体分析如下：

这个php函数用来删除文件中的重复行，还可以指定是否忽略大小写，和指定换行符

/**
 * RemoveDuplicatedLines
 * This function removes all duplicated lines of the given text file.
 *
 * @param string
 * @param bool
 * @return string
 */
function RemoveDuplicatedLines($Filepath, $IgnoreCase=false, $NewLine="\n"){
 if (!file_exists($Filepath)){
 $ErrorMsg = 'RemoveDuplicatedLines error: ';
 $ErrorMsg .= 'The given file ' . $Filepath . ' does not exist!';
 die($ErrorMsg);
 }
 $Content = file_get_contents($Filepath);
 $Content = RemoveDuplicatedLinesByString($Content, $IgnoreCase, $NewLine);
 // Is the file writeable?
 if (!is_writeable($Filepath)){
 $ErrorMsg = 'RemoveDuplicatedLines error: ';
 $ErrorMsg .= 'The given file ' . $Filepath . ' is not writeable!'; 
 die($ErrorMsg);
 }
 // Write the new file
 $FileResource = fopen($Filepath, 'w+'); 
 fwrite($FileResource, $Content); 
 fclose($FileResource); 
}
 
/**
 * RemoveDuplicatedLinesByString
 * This function removes all duplicated lines of the given string.
 *
 * @param string
 * @param bool
 * @return string
 */
function RemoveDuplicatedLinesByString($Lines, $IgnoreCase=false, $NewLine="\n"){
 if (is_array($Lines))
 $Lines = implode($NewLine, $Lines);
 $Lines = explode($NewLine, $Lines);
 $LineArray = array();
 $Duplicates = 0;
 // Go trough all lines of the given file
 for ($Line=0; $Line < count($Lines); $Line++){
 // Trim whitespace for the current line
 $CurrentLine = trim($Lines[$Line]);
 // Skip empty lines
 if ($CurrentLine == '')
 continue;
 // Use the line contents as array key
 $LineKey = $CurrentLine;
 if ($IgnoreCase)
 $LineKey = strtolower($LineKey);
 // Check if the array key already exists,
 // if not add it otherwise increase the counter
 if (!isset($LineArray[$LineKey]))
 $LineArray[$LineKey] = $CurrentLine; 
 else 
 $Duplicates++;
 }
 // Sort the array
 asort($LineArray);
 // Return how many lines got removed
 return implode($NewLine, array_values($LineArray)); 
}

使用范例：

// Example 1
// Removes all duplicated lines of the file definied in the first parameter.
$RemovedLinesCount = RemoveDuplicatedLines('test.txt');
print "Removed $RemovedLinesCount duplicate lines from the test.txt file.";
// Example 2 (Ignore case)
// Same as above, just ignores the line case.
RemoveDuplicatedLines('test.txt', true);
// Example 3 (Custom new line character)
// By using the 3rd parameter you can define which character
// should be used as new line indicator. In this case
// the example file looks like 'foo;bar;foo;foo' and will
// be replaced with 'foo;bar' 
RemoveDuplicatedLines('test.txt', false, ';');

希望本文所述对大家的php程序设计有所帮助。