All articles| All Pictures| All Softwares| All Video| Go home page| Write articles| Upload pictures

Reading number is top 10 articles
基础知识之认识和使用,RSS+asp.net_.net资料_编程技术
浅谈SQL,Server数据类型最大值_[SQL,Server教程]
VS2005+SQL2005,ASP.NET2.0数据库连接_[Asp.Net教程]
巧用VS2005解决VS2005网站发布不便问题_[Asp.Net教程]
常用的 MS SQL Server 数据修复命令_[SQL Server教程]
SQL语句在什么情况下使用全表扫描-_[SQL Server教程]
PHP正则表达式使用的方法_[PHP教程]
第六篇:ListView控件与DataPager控件详解(2)_[Asp.Net教程]
C#得到系统进程和结束某个指定的进程_.net资料_编程技术
ASP.NET技巧:下载文件关闭窗体之解决方法_.net资料_编程技术
Reading number is top 10 pictures
Athena chu perspective cheongsam shine with New York
各种囧况!玩游戏最不喜欢出现的十件事(点评)
身材野火台灣美女2
每天进步一点点
中国处女图鉴2
看到这名字我也是醉了。。。。。。
In the world the most mysterious 21 place landscape1
More attractive than sora aoi1
这才是真正的人体艺术4
In the world the most mysterious 21 place landscape3
Download software ranking
Eclipse-CALMSANNY (second edition)
功夫熊猫2(上集)
Unix video tutorial12
Unix video tutorial14
双旗镇刀客A
Unix video tutorial4
Boxer Classic video3
Jinling thirteen stock
美女写真2
Unix video tutorial13
aaa published in(发表于) 2013/12/6 10:39:08 Edit(编辑)
动态修改.Net,StreamReader,Encoding编码_.net资料_编程技术

动态修改.Net,StreamReader,Encoding编码_.net资料_编程技术

动态修改.Net StreamReader Encoding编码_.net资料_编程技术-数科优化网

在.Net framework中StreamReader的使用encoding必须在构造器中指定,而且中途完全不可以更改。


在一般的情况下,这不会造成什么问题。一般若是从硬盘读取文件,单一文件内的编码一般都是统一的。即便是发现读错,亦可以关闭StreamReader,重启使用新的编码读取。


偏偏偶最近遇到了需要修改编码的需求,而且,我的程序没有关闭重读的机会。因为偶使用的StreamReader的BaseStream是一个Network Stream,我不可以关闭它……但是Network Stream传过来的东西很可能包涵不同的编码……GB2312,Big5,UTF8,ISO-8859-1等等……虽然是先得到编码信息,然后再读具体内容,但是,一开始使用的Stream Reader编码一旦错了,读出来的东西便再也无法恢复……会丢字之类的……


我也不可以在获得编码信息之后,重新建立一个新的Stream Reader,因为具体内容已经被原来的Stream Reader给缓冲掉了……


唯一的解决方法,便是自己实现一个可以改变CurrentEncoding属性的Stream Reader了……


全部从头写起非常不实际,偶是先当了mono的源码,从mono的Stream Reader实现代码做修改。


Stream Reader其实很简单,它内部有两个Buffer,一个是input buffer,一个是decoded buffer,前者用于缓存从base stream读过来的原始数据,后者用于缓存根据原始数据解码出来后的东西……只要看明白mono的实现中ReadBuffer这个方法,要动态修改CurrentEncoding也就不是太难了……


我需要处理的网络协议是一个行协议……偶在程序中只调用了StreamReader的Readline方法,而完全没有使用Read的两个方法,这也使得偶动态修改编码容易了许多……


偶的做法是每次调用Readline的时候,不仅移动decoded buffer的游标(pos),同时也移动input buffer一个新的游标(pos_input),做法很简单,Readline方法需要调用FindNextEOL移动游标查找换行符号……我在FindNextEOL方法添加多一行:
int FindNextEOL ()
{
FindNextInputEOL();
....


而FindNextInputEOL这个新的函数,完全是FindNextEOL的翻版,只是前者处理input buffer,而后者处理decoded buffer……


如此一来,我便可以知道每次Readline之后,input buffer中还没有被上层读到的原始数据有哪些了……


然后,再把CurrentEncoding属性添加Set的方法:
set
{
encoding=value;
decoder = encoding.GetDecoder();
decoded_count = pos + decoder.GetChars (input_buffer, pos_input, cbEncoded , pos_input, decoded_buffer, pos);
}


设定新编码时,程序便根据input buffer的游标(pos_input)把没有被读到的原始数据重新decode一次,并且替换掉decoded buffer中的内容。


然后,事情就搞定了……甚至不需要对Readline方法做任何修改……除了把cbEncoded这个变量放到全局里面外……


但是,偶这个修改使得Read的两个方法变得完全不可以用……一旦调用了……便会使得input buffer与decoded buffer里面两个游标不同步……下面附上完整的代码,还望有大侠可以帮忙把Read的两个方法也给搞定了…… 先谢过……



/
// System.IO.StreamReader.cs
//
// Author:
// Dietmar Maurer (dietmar@ximian.com)
// Miguel de Icaza (miguel@ximian.com)
//
// (C) Ximian, Inc. http://www.ximian.com
// Copyright (C) 2004 Novell (http://www.novell.com)
//


//
// Copyright (C) 2004 Novell, Inc (http://www.novell.com)
//
// Permission is hereby granted, free of charge, to any person obtaining
// a copy of this software and associated documentation files (the
// "Software"), to deal in the Software without restriction, including
// without limitation the rights to use, copy, modify, merge, publish,
// distribute, sublicense, and/or sell copies of the Software, and to
// permit persons to whom the Software is furnished to do so, subject to
// the following conditions:
//
// The above copyright notice and this permission notice shall be
// included in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
// LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
// OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
// WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
//


using System;
using System.Text;
using System.Runtime.InteropServices;


namespace System.IO
{
[Serializable]
public class DynamicStreamReader : TextReader
{


const int DefaultBufferSize = 1024;
const int DefaultFileBufferSize = 4096;
const int MinimumBufferSize = 128;


//
// The input buffer
//
byte [] input_buffer;


//
// The decoded buffer from the above input buffer
//
char [] decoded_buffer;


//
// Decoded bytes in decoded_buffer.
//
int decoded_count;


//
// Current position in the decoded_buffer
//
int pos;


//
// Current position in the input_buffer
//
int pos_input;


//
// The buffer size that we are using
//
int buffer_size;


int do_checks;

Encoding encoding;
Decoder decoder;


Stream base_stream;
bool mayBlock;
StringBuilder line_builder;


private class NullStreamReader : DynamicStreamReader
{
public override int Peek ()
{
return -1;
}


public override int Read ()
{
return -1;
}


public override int Read ([In, Out] char[] buffer, int index, int count)
{
return 0;
}


public override string ReadLine ()
{
return null;
}


public override string ReadToEnd ()
{
return String.Empty;
}


public override Stream BaseStream
{
get { return Stream.Null; }
}


public override Encoding CurrentEncoding
{
get { return Encoding.Unicode; }
}
}


public new static readonly DynamicStreamReader Null = (DynamicStreamReader)(new NullStreamReader());

internal DynamicStreamReader() {}


public DynamicStreamReader(Stream stream)
: this (stream, Encoding.UTF8, true, DefaultBufferSize) { }


public DynamicStreamReader(Stream stream, bool detect_encoding_from_bytemarks)
: this (stream, Encoding.UTF8, detect_encoding_from_bytemarks, DefaultBufferSize) { }


public DynamicStreamReader(Stream stream, Encoding encoding)
: this (stream, encoding, true, DefaultBufferSize) { }


public DynamicStreamReader(Stream stream, Encoding encoding, bool detect_encoding_from_bytemarks)
: this (stream, encoding, detect_encoding_from_bytemarks, DefaultBufferSize) { }

public DynamicStreamReader(Stream stream, Encoding encoding, bool detect_encoding_from_bytemarks, int buffer_size)
{
Initialize (stream, encoding, detect_encoding_from_bytemarks, buffer_size);
}


public DynamicStreamReader(string path)
: this (path, Encoding.UTF8, true, DefaultFileBufferSize) { }


public DynamicStreamReader(string path, bool detect_encoding_from_bytemarks)
: this (path, Encoding.UTF8, detect_encoding_from_bytemarks, DefaultFileBufferSize) { }


public DynamicStreamReader(string path, Encoding encoding)
: this (path, encoding, true, DefaultFileBufferSize) { }


public DynamicStreamReader(string path, Encoding encoding, bool detect_encoding_from_bytemarks)
: this (path, encoding, detect_encoding_from_bytemarks, DefaultFileBufferSize) { }

public DynamicStreamReader(string path, Encoding encoding, bool detect_encoding_from_bytemarks, int buffer_size)
{
if (null == path)
throw new ArgumentNullException("path");
if (String.Empty == path)
throw new ArgumentException("Empty path not allowed");
if (path.IndexOfAny (Path.InvalidPathChars) != -1)
throw new ArgumentException("path contains invalid characters");
if (null == encoding)
throw new ArgumentNullException ("encoding");
if (buffer_size <= 0)
throw new ArgumentOutOfRangeException ("buffer_size", "The minimum size of the buffer must be positive");


string DirName = Path.GetDirectoryName(path);
if (DirName != String.Empty && !Directory.Exists(DirName))
throw new DirectoryNotFoundException ("Directory '" + DirName + "' not found.");
if (!File.Exists(path))
throw new FileNotFoundException("File not found.", path);


Stream stream = (Stream) File.OpenRead (path);
Initialize (stream, encoding, detect_encoding_from_bytemarks, buffer_size);
}


internal void Initialize (Stream stream, Encoding encoding, bool detect_encoding_from_bytemarks, int buffer_size)
{
if (null == stream)
throw new ArgumentNullException ("stream");
if (null == encoding)
throw new ArgumentNullException ("encoding");
if (!stream.CanRead)
throw new ArgumentException ("Cannot read stream");
if (buffer_size <= 0)
throw new ArgumentOutOfRangeException ("buffer_size", "The minimum size of the buffer must be positive");


if (buffer_size < MinimumBufferSize)
buffer_size = MinimumBufferSize;


base_stream = stream;
input_buffer = new byte [buffer_size];
this.buffer_size = buffer_size;
this.encoding = encoding;
decoder = encoding.GetDecoder ();


byte [] preamble = encoding.GetPreamble ();
do_checks = detect_encoding_from_bytemarks ? 1 : 0;
do_checks += (preamble.Length == 0) ? 0 : 2;

decoded_buffer = new char [encoding.GetMaxCharCount (buffer_size)];
decoded_count = 0;
pos = 0;
pos_input =0;
}


public virtual Stream BaseStream
{
get
{
return base_stream;
}
}


public virtual Encoding CurrentEncoding
{
get
{
if (encoding == null)
throw new Exception ();
return encoding;
}
set
{
encoding=value;
decoder = encoding.GetDecoder();
decoded_count = pos + decoder.GetChars (input_buffer, pos_input, cbEncoded - pos_input, decoded_buffer, pos);
//DiscardBufferedData();
}
}


public override void Close ()
{
Dispose (true);
}


protected override void Dispose (bool disposing)
{
if (disposing && base_stream != null)
base_stream.Close ();

input_buffer = null;
decoded_buffer = null;
encoding = null;
decoder = null;
base_stream = null;
base.Dispose (disposing);
}


//
// Provides auto-detection of the encoding, as well as skipping over
// byte marks at the beginning of a stream.
//
int DoChecks (int count)
{
if ((do_checks & 2) == 2)
{
byte [] preamble = encoding.GetPreamble ();
int c = preamble.Length;
if (count >= c)
{
int i;

for (i = 0; i < c; i++)
if (input_buffer [i] != preamble [i])
break;


if (i == c)
return i;
}
}


if ((do_checks & 1) == 1)
{
if (count < 2)
return 0;


if (input_buffer [0] == 0xfe && input_buffer [1] == 0xff)
{
this.encoding = Encoding.BigEndianUnicode;
return 2;
}


if (input_buffer [0] == 0xff && input_buffer [1] == 0xfe)
{
this.encoding = Encoding.Unicode;
return 2;
}


if (count < 3)
return 0;


if (input_buffer [0] == 0xef && input_buffer [1] == 0xbb && input_buffer [2] == 0xbf)
{
this.encoding = Encoding.UTF8;
return 3;
}
}


return 0;
}


public void DiscardBufferedData ()
{
pos = decoded_count = 0;
mayBlock = false;
// Discard internal state of the decoder too.
decoder = encoding.GetDecoder ();
}

int cbEncoded;
int parse_start;
// the buffer is empty, fill it again
private int ReadBuffer ()
{
pos = 0;
pos_input = 0;
cbEncoded = 0;


// keep looping until the decoder gives us some chars
decoded_count = 0;
parse_start = 0;
do
{
cbEncoded = base_stream.Read (input_buffer, 0, buffer_size);

if (cbEncoded == 0)
return 0;


mayBlock = (cbEncoded < buffer_size);
if (do_checks > 0)
{
Encoding old = encoding;
parse_start = DoChecks (cbEncoded);
if (old != encoding)
{
decoder = encoding.GetDecoder ();
}
do_checks = 0;
cbEncoded -= parse_start;
}
decoded_count += decoder.GetChars (input_buffer, parse_start, cbEncoded, decoded_buffer, 0);
parse_start = 0;
} while (decoded_count == 0);


return decoded_count;
}


public override int Peek ()
{
if (base_stream == null)
throw new ObjectDisposedException ("StreamReader", "Cannot read from a closed StreamReader");
if (pos >= decoded_count && (mayBlock || ReadBuffer () == 0))
return -1;


return decoded_buffer [pos];
}


public override int Read ()
{
throw new Exception("Dynamic Reader could not read!");
}


public override int Read ([In, Out] char[] dest_buffer, int index, int count)
{
throw new Exception("Dynamic Reader could not read!");
}


bool foundCR_input;
int FindNextInputEOL()
{
char c = '\0';
for (; pos_input < cbEncoded; pos_input++)
{
c = (char)input_buffer [pos_input];
if (c == '\n')
{
pos_input++;
int res = (foundCR_input) ? (pos_input - 2) : (pos_input - 1);
if (res < 0)
res = 0; // if a new buffer starts with a \n and there was a \r at
// the end of the previous one, we get here.
foundCR_input = false;
return res;
}
else if (foundCR_input)
{
foundCR_input = false;
return pos - 1;
}


foundCR_input = (c == '\r');
}


return -1;
}


bool foundCR;
int FindNextEOL ()
{
FindNextInputEOL();
char c = '\0';
for (; pos < decoded_count; pos++)
{
c = decoded_buffer [pos];
if (c == '\n')
{
pos++;
int res = (foundCR) ? (pos - 2) : (pos - 1);
if (res < 0)
res = 0; // if a new buffer starts with a \n and there was a \r at
// the end of the previous one, we get here.
foundCR = false;
return res;
}
else if (foundCR)
{
foundCR = false;
return pos - 1;
}


foundCR = (c == '\r');
}


return -1;
}


public override string ReadLine()
{
if (base_stream == null)
throw new ObjectDisposedException ("StreamReader", "Cannot read from a closed StreamReader");


if (pos >= decoded_count && ReadBuffer () == 0)
return null;


int begin = pos;
int end = FindNextEOL ();
if (end < decoded_count && end >= begin)
return new string (decoded_buffer, begin, end - begin);


if (line_builder == null)
line_builder = new StringBuilder ();
else
line_builder.Length = 0;


while (true)
{
if (foundCR) // don't include the trailing CR if present
decoded_count--;


line_builder.Append (new string (decoded_buffer, begin, decoded_count - begin));
if (ReadBuffer () == 0)
{
if (line_builder.Capacity > 32768)
{
StringBuilder sb = line_builder;
line_builder = null;
return sb.ToString (0, sb.Length);
}
return line_builder.ToString (0, line_builder.Length);
}


begin = pos;
end = FindNextEOL ();
if (end < decoded_count && end >= begin)
{
line_builder.Append (new string (decoded_buffer, begin, end - begin));
if (line_builder.Capacity > 32768)
{
StringBuilder sb = line_builder;
line_builder = null;
return sb.ToString (0, sb.Length);
}
return line_builder.ToString (0, line_builder.Length);
}
}
}


public override string ReadToEnd()
{
if (base_stream == null)
throw new ObjectDisposedException ("StreamReader", "Cannot read from a closed StreamReader");


StringBuilder text = new StringBuilder ();


int size = decoded_buffer.Length;
char [] buffer = new char [size];
int len;

while ((len = Read (buffer, 0, size)) > 0)
text.Append (buffer, 0, len);


return text.ToString ();
}
}
}






添加到del.icio.us 添加到新浪ViVi 添加到百度搜藏 添加到POCO网摘 添加到天天网摘365Key 添加到和讯网摘 添加到天极网摘 添加到黑米书签 添加到QQ书签 添加到雅虎收藏 添加到奇客发现 diigo it 添加到饭否 添加到飞豆订阅 添加到抓虾收藏 添加到鲜果订阅 digg it 貼到funP 添加到有道阅读 Live Favorites 添加到Newsvine 打印本页 用Email发送本页 在Facebook上分享


Disclaimer Privacy Policy About us Site Map

If you have any requirements, please contact webmaster。(如果有什么要求,请联系站长)
Copyright ©2011-
uuhomepage.com, Inc. All rights reserved.