Ñêðèïò äëÿ ñêà÷èâàíèÿ ïîäêàñòîâ Python + Google Reader

[1ac3124]

Êîãäà êîëè÷åñòâî ïðîñëóøèâàåìûõ ïîäêàñòîâ ïåðåâàëèëî çà 3 ñòàëî äîñòàòî÷íî ñëîæíî ñëåäèòü çà íèìè âñåìè. Ïîëüçîâàòüñÿ àãðåãàòîðàìè ìîíñòðàìè òèïà iTunes íåò æåëàíèÿ, äà è ïðîáëåìû ñ íèìè òîæå âîçíèêàþò: òî â çàáóäó òûêíóòü êíîïî÷êó îáíîâèòü/ñêà÷àòü, òî âðåìåíè íå õâàòàåò äîæäàòüñÿ ñêà÷èâàíèÿ.  îáùåì ïðîáëåìà àâòîìàòè÷åñêîé äîñòàâêè ïîäêàñò-êîíòåíòà ïîáëèæå êî ìíå âèäíà íà ëèöî. Ðåøåíèå ïðîáëåìû ïðèíÿëñÿ èñêàòü êàðäèíàëüíî, íî óäîáíîå.  èòîãå íà áûë íàéäåí ïèòîíîâñêèé ñêðèïò äëÿ àâòîìàòè÷åñêîãî ñêà÷èâàíèÿ ïîäêàñòîâ èç ëåíòû Google Reader. Äåëî îñòàëîñü çà ìàëûì: çàïóñòèòü âñ¸ ýòî íà ðîóòåðå.

Ñïèñîê ïîäêàñòåðñêèõ ëåíò õðàíèì Google Reader (http://www.google.ru/reader/). Ëåíòû ïîìå÷åíû ñâîèì òåãîì, è àêêóðàòíî ëåæàò â ñâîåé ïàïêå.  êà÷åñòâå îñíîâû ñêðèïòà âçÿòà áèáëèîòåêó pyrfeed (http://code.google.com/p/pyrfeed/), â êîòîðîé ðåàëèçîâàí ïîëåçíûé êëàññ GoogleReader.

1. Ñíà÷àëà ñòàâèì Python (íà âðåìÿ íàïèñàíèÿ ñòàòüè òåêóùàÿ âåðñèÿ â ðåïîçèòîðèè 2.5):

ipkg install python
ipkg install py25-elementtree
ipkg install py25-cheetah
ipkg install py25-cherrypy
2. Ðàñïàêîâûâàåì pyrfeed-0.5.1.rar (http://1ac3124.tzo.org/pyrfeed-0.5.1.rar) â /opt/lib/python2.5

3. Ñîäåðæèìîå PodcastDownloadTool.rar (http://1ac3124.tzo.org/PodcastDownloadTool.rar) ðàñïàêîâûâàåì â ëþáîå óäîáíîå âàì ìåñòî, íàïðèìåð /opt/urs/bin/

4. Íàñòðàèâàåì ñêðèïò PodcastDownloadTool.py ïîä ñåáÿ:
downloadDir – êàòàëîã, êóäà áóäóò ñêà÷èâàòüñÿ ïîäêàñòû
logFile – íàçâàíèå ëîã ôàéëà
tag – èìÿ òåãà/ïàïêè â Google Reader, â êîòîðîé áóäóò ïðîñìàòðèâàòüñÿ ëåíòû
login è password – ëîãèí è ïàðîëü â Google Reader

5. Ïðîâåðÿåì ðàáîòîñïîñîáíîñòü:

python /opt/usr/bin/PodcastDownloadTool.py
 îòâåò ìû äîëæíû ïîëó÷èòü êàê ìèíèìóì "Login OK", åñëè åñòü íåïðî÷èòàííûå ëåíòû ïîäêàñòîâ, òî íà÷í¸òñÿ èõ ñêà÷èâàíèå.

6. Îñòàëîñü çàñòàâèòü âûïîëíÿòü ðîóòåð çàïóñêàòü ñêðèïò ðàç â ÷àñ/äåíü êîìó êàê óäîáíî. Ñîçäàåì â /opt/etc/cron.hourly (èëè /opt/etc/cron.daily) ñêðèïò (íàïðèìåð PodcastDownloadTool.sh) ñëåäóþùåãî ñîäåðæàíèÿ:

#!/bin/sh
python /opt/usr/bin/PodcastDownloadTool.py >> /opt/var/log/PodcastDownloadTool.log 2>&1

Íà ýòîì ñîáñòâåííî âñ¸, îñòàëîñü óêàçàòü íà ìàëåíüêèå íåäîñòàòêè:

Íå ïîääåðæèâàåòñÿ äîêà÷êà.  ñëó÷àå ñáîÿ RSS-çàïèñü íå áóäåò ïîìå÷åíà êàê ïðî÷èòàííàÿ. Ïðè ñëåäóþùåì çàïóñêå ñêðèïòà ôàéë áóäåò âûêà÷àí çàíîâî.
Îòñþäà ñëåäóþùèé ìîìåíò – ïðè ñëåäóþùåì çàïóñêå ñêðèïòà, îøèáî÷íûå çàïèñè (íàïðèìåð, â çàïèñè íåò ññûëêè íà ôàéë äëÿ ñêà÷èâàíèÿ) áóäóò ïîâòîðíî îáðàáàòûâàòüñÿ. Ìîæíî áû èõ ìåòèòü ñïåöèàëüíûì òåãîì è ïðîïóñêàòü â äàëüíåéøåì. Íî ìåíÿ óñòðàèâàåò ïîñëåäóþùèé ðó÷íîé ïðîñìîòð ëåíòû íà ïðåäìåò íå ïðî÷èòàííûõ çàïèñåé.
Õðàíåíèå êîíôèãóðàöèè – ëîãèí è ïàðîëü â îòêðûòîì âèäå. Ëîãèí åùå íå òàê ñòðàøíî, à âîò ïàðîëü (åñëè ó êîãî-òî ïàðàíîéÿ, òî ìîæíî ðàäè ýòîãî çàâåñòè ïóñòîãî ïîëüçîâàòåëÿ â GoogleReader)



Ìàíóàë íàïèñàí íà îñíîâå ñòàòüè èç áëîãà Yustos (http://yustos.ru/blog/podcast-downloading-with-python-and-google-reader).

[kibenimatik]

Çäîðîâî. ß êàê ðàç äóìàë î òîì êàê îðãàíèçîâàòü àâòîìàòè÷åñêóþ çàãðóçêó ïðîãðàììû òåëåïåðåäà÷. Ñïàñèáî çà íàâîäêó.

[RekoD]

Ïðîáîâàë ïîäíÿòü proxymin íà WL500gPV2 (íà x86 ðàáîòàåò íà óðà), íî ïîëó÷èë ñëåäóþùåå:


--> -->


DBInvalidArgError Python 2.4.5: /opt/bin/python2.4
Tue Jan 6 00:59:27 2009

A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.
/opt/share/www/ctrl/proxymin.py
91 # call appropriate page
92 if(action == ACTION_SHOW_MAIN):
93 gui_main.print_main_page(category, letter, group)
94 elif(action == ACTION_DISPLAY_HELP):
95 gui_help.print_help_page()
gui_main = <module 'gui_main' from '/opt/share/www/ctrl/gui_main.py'>, gui_main.print_main_page = <function print_main_page>, category = 'groups', letter = None, group = None
/opt/share/www/ctrl/gui_main.py in print_main_page(category='groups', letter=None, group=None)
30 """display administration main page"""
31 gui_misc.print_html_head()
32 print_title_bar()
33 print '<br><br>'
34 print_tabs(category)
global print_title_bar = <function print_title_bar>
/opt/share/www/ctrl/gui_main.py in print_title_bar()
51 def print_title_bar():
52 # count all user/host/group objects
53 iterator = objectiterator.ObjectIterator(CATEGORY_HOSTS)
54 number_of_hosts = iterator.get_object_count()
55 iterator.set_category(CATEGORY_USERS)
iterator undefined, global objectiterator = <module 'objectiterator' from '/opt/share/www/ctrl/objectiterator.py'>, objectiterator.ObjectIterator = <class objectiterator.ObjectIterator>, global CATEGORY_HOSTS = 'hosts'
/opt/share/www/ctrl/objectiterator.py in __init__(self=<objectiterator.ObjectIterator instance>, category='hosts')
25 """uniform interface to access different object types - simplifies GUI programming"""
26 def __init__(self, category):
27 self.dba = dbaccess.DBAccess()
28 self.set_category(category)
29
self = <objectiterator.ObjectIterator instance>, self.dba undefined, global dbaccess = <module 'dbaccess' from '/opt/share/www/ctrl/dbaccess.py'>, dbaccess.DBAccess = <class dbaccess.DBAccess>
/opt/share/www/ctrl/dbaccess.py in __init__(self=<dbaccess.DBAccess instance>)
39 """persistence framework"""
40 def __init__(self):
41 self.users = shelve.open(userdb)
42 self.hosts = shelve.open(hostdb)
43 self.groups = shelve.open(groupdb)
self = <dbaccess.DBAccess instance>, self.users undefined, global shelve = <module 'shelve' from '/opt/lib/python2.4/shelve.pyc'>, shelve.open = <function open>, global userdb = 'db//users'
/opt/lib/python2.4/shelve.py in open(filename='db//users', flag='c', protocol=None, writeback=False, binary=None)
230
231 See the module's __doc__ string for an overview of the interface.
232 """
233
234 return DbfilenameShelf(filename, flag, protocol, writeback, binary)
global DbfilenameShelf = <class shelve.DbfilenameShelf>, filename = 'db//users', flag = 'c', protocol = None, writeback = False, binary = None
/opt/lib/python2.4/shelve.py in __init__(self=<DbfilenameShelf instance>, filename='db//users', flag='c', protocol=None, writeback=False, binary=None)
213 def __init__(self, filename, flag='c', protocol=None, writeback=False, binary=None):
214 import anydbm
215 Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback, binary)
216
217
global Shelf = <class shelve.Shelf>, Shelf.__init__ = <unbound method Shelf.__init__>, self = <DbfilenameShelf instance>, anydbm = <module 'anydbm' from '/opt/lib/python2.4/anydbm.pyc'>, anydbm.open = <function open>, filename = 'db//users', flag = 'c', protocol = None, writeback = False, binary = None
/opt/lib/python2.4/anydbm.py in open(file='db//users', flag='c', mode=438)
79 # db type cannot be determined
80 raise error, "db type could not be determined"
81 else:
82 mod = __import__(result)
83 return mod.open(file, flag, mode)
mod = <module 'dbhash' from '/opt/lib/python2.4/dbhash.pyc'>, mod.open = <function open>, file = 'db//users', flag = 'c', mode = 438
/opt/lib/python2.4/dbhash.py in open(file='db//users', flag='c', mode=438)
12
13 error = bsddb.error # Exported for anydbm
14
15 def open(file, flag = 'r', mode=0666):
16 return bsddb.hashopen(file, flag, mode)
global bsddb = <module 'bsddb' from '/opt/lib/python2.4/bsddb/__init__.pyc'>, bsddb.hashopen = <function hashopen>, file = 'db//users', flag = 'c', mode = 438
/opt/lib/python2.4/bsddb/__init__.py in hashopen(file='db//users', flag='c', mode=438, pgsize=None, ffactor=None, nelem=None, cachesize=None, lorder=None, hflags=0)
289
290 flags = _checkflag(flag, file)
291 e = _openDBEnv(cachesize)
292 d = db.DB(e)
293 d.set_flags(hflags)
e undefined, global _openDBEnv = <function _openDBEnv>, cachesize = None
/opt/lib/python2.4/bsddb/__init__.py in _openDBEnv(cachesize=None)
345 else:
346 raise error, "cachesize must be >= 20480"
347 e.open('.', db.DB_PRIVATE | db.DB_CREATE | db.DB_THREAD | db.DB_INIT_LOCK | db.DB_INIT_MPOOL)
348 return e
349
e = <DBEnv object>, e.open = <built-in method open of DBEnv object>, global db = <module '_bsddb' from '/opt/lib/python2.4/lib-dynload/_bsddb.so'>, db.DB_PRIVATE = 1048576, db.DB_CREATE = 1, db.DB_THREAD = 64, db.DB_INIT_LOCK = 8192, db.DB_INIT_MPOOL = 32768

DBInvalidArgError: (22, 'Invalid argument -- architecture lacks fast mutexes: applications cannot be threaded')
args = (22, 'Invalid argument -- architecture lacks fast mutexes: applications cannot be threaded')

Ò.å. íå ïîääåðæèâàþòñÿ êàêèå òî fast mutexes è ïîýòîìó proxymin íå ìîæåò îòêðûòü ñâîþ ëîêàëüíóþ áàçó äàííûõ.

Êòî-íèáóäü çíàåò - ëå÷èòñÿ ëè ýòî êàê íèáóäü?

[zina79]

Àâòîð, åñëè íå òðóäíî, îáíîâèòå ññûëêè äëÿ ñêà÷èâàíèÿ. Ñïàñèáî.:)

[Delirium]

Всем привет.

Роутер wl500g подключен непосредственно к провайдеру и имеет выделенный статический IP. thttpd сервер работает отлично как на LAN так и на WAN.

Решил попробывать запустить на роутер простой HTTP сервер написанный на python. Вот его код:



from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler

import shutil
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO

class MyHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):

f = StringIO()
f.write('<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">')
f.write("<html>\n<title>Test Page</title>\n")
f.write("<body>\n<h2>Hi</h2>\n")
f.write("</body></html>\n")

length = f.tell()
self.send_response(200)
self.send_header("Content-type", "text/html")
self.send_header("Content-Length", str(length))
self.end_headers()
f.seek(0)

self.wfile.write(f.read(length))


server_address = ('domain_name', 8666)
server = HTTPServer(server_address, MyHTTPRequestHandler)

while 1 == 1:
server.handle_request()


iptables -I INPUT 7 -p tcp --dport 8666 -j ACCEPT
прописано.

Он запускается и работает отлично в LAN. Но стоит зайти на него с WAN, всё сразу падает с Segmentation fault:



[delirium@Delirium_Router disc0_1]$ python server.py
92.241.190.197 - - [18/Aug/2010 15:16:17] "GET / HTTP/1.1" 200 -
92.241.190.197 - - [18/Aug/2010 15:16:20] "GET /favicon.ico HTTP/1.1" 200 -
92.241.190.197 - - [18/Aug/2010 15:17:00] "GET / HTTP/1.1" 200 -
92.241.190.197 - - [18/Aug/2010 15:17:03] "GET /favicon.ico HTTP/1.1" 200 -
Segmentation fault

(Первые два запроса из LAN, затем из WAN)

Версия python:
Python 2.5.5 (r255:77872, May 20 2010, 23:42:58)
[GCC 4.1.1] on linux2

Версия прошивки роутера: Olegs firmware 1.9.2.7-10

С чем это может быть связано? Куда копать?