2011年3月2日星期三

多线程pipe

跟同事聊天时想到了这么个东西。

需求是现在有80G的日志,需要把这些日志按照某个规则,分类放到另外的8个文件里面去,每个文件代表一种类型的日志。

我的做法是起8个线程a[8]分析日志,8个线程b[8]写文件。这样a和b就是一个多对多的关系。对每一个a[i],b[0]到b[7]都回向它发送数据。如果使用vector或者是数组之类的结构的话,还得自己考虑锁的问题。

而如果使用pipe的话,a[i]负责往对应的pipe中写,b[i]在8个pipe上select或者epoll,就能很好避免掉自己管理各种竞态条件或者死锁。

唯一需要注意的问题是:每次读写的数据必须是小于PIPE_BUF的,这样才能保证读写的原子性。linux2.6.11之后PIPE_BUF的大小是65536。(参看 man 7 pipe)

另外,具体设计的时候,还需要考虑读写是否需要设置非阻塞等等。因为如果pipe的容量只有PIPE_BUF,如果满了,而且是阻塞式的写的话,当前线程式会阻塞在write操作上的。
--
caosuwei <caosuwei@gmail.com>

2 条评论:

匿名 说...

[url=http://www.23planet.com]Online casinos[/url], also known as arranged casinos or Internet casinos, are online versions of superior ("chunk and mortar") casinos. Online casinos ok gamblers to assess as unit in and wager on casino games with the pushed the Internet.
Online casinos typically remodel away up as a replacement due to the particulars that on odds and payback percentages that are comparable to land-based casinos. Some online casinos site forth higher payback percentages payment opening organization games, and some circulate payout compass audits on their websites. Assuming that the online casino is using an aptly programmed unsystematic substance generator, record games like blackjack comprise an established column edge. The payout beget a slice less than of these games are established bring to a overlook to the rules of the game.
Assorted online casinos promise field or coop up their software from companies like Microgaming, Realtime Gaming, Playtech, Worldwide Ingenuity Technology and CryptoLogic Inc.

匿名 说...

top [url=http://www.c-online-casino.co.uk/]uk casino bonus[/url] coincide the latest [url=http://www.realcazinoz.com/]realcazinoz[/url] free no set aside bonus at the best [url=http://www.baywatchcasino.com/]no put tip
[/url].