VSIDO Community

VSIDO Support => General Support => Topic started by: VastOne on November 12, 2016, 10:21:36 PM

Title: gmail-parser.py failure
Post by: VastOne on November 12, 2016, 10:21:36 PM
Issue with gmail-parser.py script I have been running forever...

I recently had to reinstall my main build system and in the rebuild, I cannot get the gmail-parser.py script to run anymore.. It still works fine on all other machines so it is definitely something I have not installed (something I have forgotten...)

Here is the conky feedback when it is run

  File "/home/vastone/gmail_parser.py", line 46, in <module>
    f = auth()  # Do auth and then get the feed
  File "/home/vastone/gmail_parser.py", line 30, in auth
    feed = f.read()
  File "/usr/lib/python2.7/socket.py", line 355, in read
    data = self._sock.recv(rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 766, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 653, in read
    v = self._sslobj.read(len)
socket.error: [Errno 0] Error


I have everything the script installed (feedparser etc) but no go to get it to run

Anyone spot what I am missing?
Title: Re: gmail-parser.py failure
Post by: PackRat on November 12, 2016, 10:44:19 PM
As I recall there are 2-3 dependencies for that script - one was called py-parser, python-parser or something like that and pulled in the other 2.

I vaguely remember reading a thread about this where you solved it - you have the old #! forum bookmarked? If you search that, you may find your old thread.

this maybe -

gmail python script (http://crunchbang.org/forums/viewtopic.php?id=381)

python-feedparser maybe
Title: Re: gmail-parser.py failure
Post by: VastOne on November 12, 2016, 10:47:04 PM
Yes.. I wrote a How To over there for it... :D

The needed app is python-feedparser and that is correctly installed... everything looks as if it should function fine but I am cannot get it to go..

Title: Re: gmail-parser.py failure
Post by: PackRat on November 12, 2016, 10:48:07 PM
beat me to it  :D :D

Clean [re]-install of your system?

Next guess would be you need python 2.7 but Debian in python 3 by default now.
Title: Re: gmail-parser.py failure
Post by: VastOne on November 12, 2016, 10:53:46 PM
Yes... a clean install

But 2.7 and 3.0 are both still installed as default although I did run vsido-welcome so it could have been installed during that but I do not believe so
Title: Re: gmail-parser.py failure
Post by: VastOne on November 12, 2016, 10:57:50 PM
I have also spent most of today google-fu the fuck out of this issue and have not been able to resolve...

I have found a nice bash alternative that just shows me I have new emails (count)

If we solve it fine, if not fine.... :D
Title: Re: gmail-parser.py failure
Post by: PackRat on November 12, 2016, 11:24:50 PM
Does your gmail-parser.py run in a terminal?

Might be a ...wait for it ... conky issue.

I've had conky barf on some older configs after doing a reinstall, usually it will not run a bash script. A couple times I rewrote the conky config.

Should be conky 1.10 on your new install, correct? The other machines conky 1.9 or 1.10? And is your conkyrc in 1.10 format?
Title: Re: gmail-parser.py failure
Post by: VastOne on November 12, 2016, 11:26:56 PM
Same conky on all systems...

Good thought... :D
Title: Re: gmail-parser.py failure
Post by: jedi on November 13, 2016, 03:32:08 AM
Hey VastOne!  A Cub's world series after all...   8)

I've noticed with a couple of different python scripts, you need to either specify in the code that it is indeed python2, or start it with "python2" instead of just "python".  Give it a try and good luck!  And yes both Pythons are installed in VSIDO by default.  You might also need to add it to the $PATH...  ;D
Title: Re: gmail-parser.py failure
Post by: VastOne on November 13, 2016, 03:39:55 AM
Same failures running it with python2 as with regular python

Using python3 gets me further but syntax errors pop up with it, which means more than anything the script needs a rewrite

I am only saddened by this as I have been using this particular script for as long as I have been using Linux... quite a long time

Yes.. the Cubs are legit...
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 13, 2016, 02:14:51 PM
Just make sure the following packages are there (they should be, most likely),


openssl
python-openssl
python3-openssl


If they are there and/or installing them doesn't solve the problem then open 'gmail-parser.py' in text-editor and add either of the following snippets at the top of the script,


import ssl
ssl.PROTOCOL_SSLv23 = ssl.PROTOCOL_TLSv1



import functools
import ssl

old_init = ssl.SSLSocket.__init__

@functools.wraps(old_init)
def vsido_gmail_bug(self, *args, **kwargs):
  kwargs['ssl_version'] = ssl.PROTOCOL_TLSv1
  old_init(self, *args, **kwargs)

ssl.SSLSocket.__init__ = vsido_gmail_bug


The first one is a simple one while the second a bit more elegant! One of them should work, Hopefully  ;)

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 13, 2016, 04:22:32 PM
Thank you hakerdefo for the attempt... but these all failed too..

:(

Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 13, 2016, 09:23:43 PM
Hi there VastOne,
Please run the gmail-parser.py script in terminal and post the complete output here.
Cheers!!!
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 13, 2016, 09:25:02 PM
And please post the gmail-parser.py script in question as well!
Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 13, 2016, 09:30:51 PM
python 2 output:

Traceback (most recent call last):
  File "/home/vastone/gmail_parser.py", line 22, in <module>
    maxlen = sys.argv[3]
IndexError: list index out of range


python 3 is a different error but am posting for knowledge:

File "/home/vastone/gmail_parser.py", line 37
    print '${color3} %s new email(s)\n' % (len(atom.entries))
                                      ^
SyntaxError: invalid syntax


gmail_parser.py

## check-gmail.py -- A command line util to check GMail -*- Python -*-
## modified to display mailbox summary for conky

# ======================================================================
# Copyright (C) 2006 Baishampayan Ghose <b.ghose@ubuntu.com>
# Modified 2008 Hunter Loftis <hbloftis@uncc.edu>
# Time-stamp: Mon Jul 31, 2006 20:45+0530
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
# ======================================================================

import sys
import urllib             # For BasicHTTPAuthentication
import feedparser         # For parsing the feed
from textwrap import fill

_URL = "https://mail.google.com/gmail/feed/atom"

uname = sys.argv[1]
password = sys.argv[2]
maxlen = sys.argv[3]

urllib.FancyURLopener.prompt_user_passwd = lambda self, host, realm: (uname, password)

def auth():
    '''The method to do HTTPBasicAuthentication'''
    opener = urllib.FancyURLopener()
    f = opener.open(_URL)
    feed = f.read()
    return feed


def readmail(feed, maxlen):
'''Parse the Atom feed and print a summary'''
atom = feedparser.parse(feed)
print '${color3} %s new email(s)\n' % (len(atom.entries))
for i in range(min(len(atom.entries), maxlen)):
print '${color2}%s' % fill(atom.entries[i].title,38)
#uncomment the following line if you want to show the name of the sender
# print '          ${color2}%s' % atom.entries[i].author
if len(atom.entries) > maxlen:
print ' ${color}more...'

if __name__ == "__main__":
    f = auth()  # Do auth and then get the feed
    readmail(f, int(maxlen)) # Let the feed be chewed by feedparser


Run it as

python /home/location_of_file/gmail_parser.py yourusername yourpassword
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 13, 2016, 09:40:09 PM
How about a shot in the dark  ;)


sudo apt-get install python-feedparser


Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 13, 2016, 09:52:53 PM
As stated above, I actually wrote the how to's on this thing..

Yes, it and python3-feedparser is installed
Title: Re: gmail-parser.py failure
Post by: PackRat on November 14, 2016, 12:58:01 AM
Doesn't happen to be a python2-feedparser used by your other installs, does there?

And I get identical error messages using python, python2.7 and python3.5 in

python /home/location_of_file/gmail_parser.py yourusername yourpassword

running the script on Void linux, python-feedparser version 5.2.1
Title: Re: gmail-parser.py failure
Post by: VastOne on November 14, 2016, 02:34:42 AM
Nope..  Just the latest SID version..  5.1.3-3
Title: Re: gmail-parser.py failure
Post by: jedi on November 14, 2016, 06:24:31 AM
not that it's any comfort, but mine is the exact same way. same error, tried all of above and no joy...
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 14, 2016, 05:55:56 PM
I think I've figured this one! Warning, I've been proved wrong before  ;)
VastOne, Time has come to update this script cause changes made in python and urllib has rendered it obsolete! Are you sure gmail-parser.py is working on other Debian Sid installs? Anyways here is the modification of gmail-parser.py that should hopefully work with python 3 & co.

#! /usr/bin/env python3

import urllib.request
import urllib             # For BasicHTTPAuthentication
import feedparser         # For parsing the feed
from textwrap import wrap # For pretty printing assistance
import sys
import time

_URL = "https://mail.google.com/gmail/feed/atom/unread"
WRAP_LIMIT = 50

def auth():

    username = "your_gmail_username_here"
    password = "your_gmail_password_here"
   
    auth_handler = urllib.request.HTTPBasicAuthHandler()
    auth_handler.add_password(realm='New mail feed',
                              uri='https://mail.google.com/',
                              user= username,
                              passwd= password)

    opener = urllib.request.build_opener(auth_handler)
    # ...and install it globally so it can be used with urlopen.
    urllib.request.install_opener(opener)
   
    '''The method to do HTTPBasicAuthentication'''
   
    f = opener.open(_URL)
    feed = f.read()
    return feed

def fill(text, width):
    '''A custom method to assist in pretty printing'''
    if len(text) < width:
        return text + ' '*(width-len(text))
    else:
        return text

def readmail(feed):
    '''Parse the Atom feed and print a summary'''
    atom = feedparser.parse(feed)

    print ("${color white}You have %s new mails${color} ${alignr}Updated: ${color white}%s" % ((len(atom.entries)), time.strftime("%I:%M")))

    for i in range(len(atom.entries)):
        if(i>10):
            break
        if(len(atom.entries[i].title) > WRAP_LIMIT):
        #print ("%s" % (fill(wrap(atom.entries[i].title, 50)[0]+" ...", 55)))
            print ("${color1}%s" % (wrap(atom.entries[i].title, WRAP_LIMIT)[0]+" ..."))
        else:
            print ("${color1}%s" % (wrap(atom.entries[i].title, WRAP_LIMIT)[0]))

def countmail(feed):
    '''Parse the Atom feed and print a summary'''
    atom = feedparser.parse(feed) 
    print ("Emails: %s new" %len(atom.entries))
               
if __name__ == "__main__":
    f = auth()  # Do auth and then get the feed
    if(len(sys.argv) > 1):
        countmail(f)
    else:
        readmail(f) # Let the feed be chewed by feedparse



You will need to add your Gmail username and password to the following fields in the script before using it,


username = "your_gmail_username_here"
password = "your_gmail_password_here"


Save it somewhere in your $PATH and call it like this from conky,


execpi 600 /path/to/gmail-parser.py


Of-course you will have to replace '/path/to/gmail-parser.py' above with the actual path where you have saved the script.

Absolutely untested as I don't have python 3 on this box! Might blow up your computer! You've been warned! Good luck!

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: PackRat on November 14, 2016, 07:40:19 PM
I get an error running it in a terminal -

Traceback (most recent call last):
  File "./gmail-parser.py", line 5, in <module>
    import feedparser         # For parsing the feed
ImportError: No module named 'feedparser'


@vastone - python-feedparser the only depndency?
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 14, 2016, 07:50:01 PM
Hi there PackRat,
You will need 'python3-feedparser' for the modified script I've posted. So just do,


sudo apt-get install python3-feedparser


And things should be fine!

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: PackRat on November 14, 2016, 09:26:22 PM
Ok, I get a whole bunch of new errors -

Traceback (most recent call last):
  File "./gmail-parser.py", line 62, in <module>
    f = auth()  # Do auth and then get the feed
  File "./gmail-parser.py", line 30, in auth
    f = opener.open(_URL)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 504, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized


This system is Void linux - I can check your script on my Debian system once the crew heads for Girl Scouts and the desk is free.
Title: Re: gmail-parser.py failure
Post by: VastOne on November 15, 2016, 12:02:07 AM
Similar (if not same as PackRat) errors

Traceback (most recent call last):
  File "/home/vastone/gmail_parser.py", line 62, in <module>
    f = auth()  # Do auth and then get the feed
  File "/home/vastone/gmail_parser.py", line 30, in auth
    f = opener.open(_URL)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 504, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 693, in http_error_302
    fp.read()
  File "/usr/lib/python3.5/http/client.py", line 458, in read
    s = self.fp.read()
  File "/usr/lib/python3.5/socket.py", line 576, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.5/ssl.py", line 937, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.5/ssl.py", line 799, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.5/ssl.py", line 583, in read
    v = self._sslobj.read(len, buffer)
OSError: [Errno 0] Error
Title: Re: gmail-parser.py failure
Post by: PackRat on November 15, 2016, 03:18:25 AM
@vastone - you have a link to your #! HowTo? I didn't see it recreated here. Void is crazy minimal so I need to make sure I have the packages installed - there no recommneded packages with Void just hard dependencies (wget isn't even part of the default install - distro is bat-shit crazy in some ways).
Title: Re: gmail-parser.py failure
Post by: VastOne on November 15, 2016, 03:31:51 AM
There is a How To on #! but I'll be damned if I can find it there..

The earliest references I have found of a How To on this by me is here on the Ubuntu forums in a thread about conky (https://ubuntuforums.org/showthread.php?t=281865&page=1352&p=9721860#post9721860)

The only requirement though is and has always been python-feedparser, nothing else
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 15, 2016, 06:08:33 AM
@PackRat @VastOne
You've got following installed?


python3-openssl
python3-requests
python3-urllib3


Install if missing and re-run the script!

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: jedi on November 15, 2016, 06:41:17 AM
Traceback (most recent call last):
  File "/home/jedi/conky/gmail_parser.py", line 23, in <module>
    maxlen = sys.argv[3]
IndexError: list index out of range



Still the same...
Title: Re: gmail-parser.py failure
Post by: Snap on November 15, 2016, 07:19:43 AM
Folks, AFAIK there's currently something libssl breaking python. Might be that driving you mad?
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 15, 2016, 09:14:38 AM
@Snap is right! Screw Python for now and switch to Ruby  8)

First make sure following packages are installed,


ruby
libruby


Gmail by default blocks all non-Google applications from accessing your account. There are two things you can do here to make this script work with Gmail.
First Method (recommended) :-
Generate app password for this script and use that generated password in the script. More information here,

Sign in using App Passwords (https://support.google.com/accounts/answer/185833?hl=en)
Second Method :-
Change your Goolge account setting to allow non-Google apps access to your account. More information here,
Allowing less secure apps to access your account (https://support.google.com/accounts/answer/6010255?hl=en)

Next copy the following script with the name 'check-gmail.rb' somewhere in your $PATH,


#!/usr/bin/ruby
require 'net/imap'
IMAP_USER = "your_gmail_username"
IMAP_PASS = "your_gmail_password"
IMAP_SERVER = "imap.gmail.com"
SSL = true
PORT = 993
IS_EXCHANGE = false
IMAP_FOLDER = "INBOX"
CONKYFIED = true
NEW_COLOR = "green"
SENDER_COLOR = "grey"
SUBJ_COLOR = "white"

def conkify(text,color)
    text = "${color #{color}}#{text} $color"
    return text
end

def get_messages(conn)
    imap = conn
    imap.search(["NOT", "DELETED", "UNSEEN"]).each do |message_id|
        envelope = imap.fetch(message_id, "ENVELOPE")[0].attr["ENVELOPE"]
        from, subject = envelope.from[0].name == nil ? envelope.from[0].mailbox : envelope.from[0].name, envelope.subject
        CONKYFIED == true ? puts("#{conkify(from,SENDER_COLOR)} $alignr#{conkify(subject,SUBJ_COLOR)}") : puts("#{from}  #{subject}")
    end
end

imap = Net::IMAP.new(IMAP_SERVER,port = PORT, usessl = SSL)
imap.login(IMAP_USER,IMAP_PASS)
imap.examine(IMAP_FOLDER)
IS_EXCHANGE == true ? counts = imap.search(["NOT", "DELETED", "UNSEEN"]).size : counts = imap.status(IMAP_FOLDER, ["UNSEEN"])["UNSEEN"]
counts > 0 ? ( puts("New Messages: #{CONKYFIED == true ? conkify(counts,NEW_COLOR) : counts}"); get_messages(imap) ) : ( puts("No new messages") )
imap.disconnect
exit



You'll need to modify the following in the script before using it,


IMAP_USER = "your_gmail_username"
IMAP_PASS = "your_gmail_password"


To modify the color output of the script shown in conky, modify the following fields to your liking in the script,


NEW_COLOR = "green"
SENDER_COLOR = "grey"
SUBJ_COLOR = "white"


If you wish to use the script in terminal instead of conky, change the following in the script from,


CONKYFIED = true


to


CONKYFIED = false


Make sure to give the script correct permissions,


chmod 755 /path/to/check-gmail.rb


And call it like following from conky,


execpi 600 /path/to/check-gmail.rb


This is tested okay & satisfaction guaranteed!!!

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: PackRat on November 15, 2016, 12:52:58 PM
Quote from: hakerdefo on November 15, 2016, 06:08:33 AM
@PackRat @VastOne
You've got following installed?


python3-openssl
python3-requests
python3-urllib3


Install if missing and re-run the script!

Cheers!!!

No change
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 15, 2016, 09:06:36 PM
Time to put this thread to rest! Why? Cause my friend Snap could be right when he said,

Quote from: Snap on November 15, 2016, 07:19:43 AM
Folks, AFAIK there's currently something libssl breaking python. Might be that driving you mad?

What's the proof I hear you say?

Well, I fired-up Semplice workstation (based on Debian stable) in live session. Took the 'check-gmail.py' posted earlier in this thread by VastOne and installed following packages,


python-urllib3
python-requests
python-feedparser


And run the 'check-gmail.py' script thus,


python check-gmail.py "gmail_username" "gmail_password" "3"


The result? Here you go,

(https://cdn.scrot.moe/images/2016/11/15/screenshot_2016-11-15-153655.md.png) (https://scrot.moe/image/uui2)

The username & password are hidden in the image for obvious reasons but you know what? It Fu***ng works  :o

Now why oh why this doesn't work in Sid? As Snap mentioned there could be some bug out there somewhere in the Python land that's why!
What to do in the mean time? Use the ruby script I posted previously. It's small, doesn't have any dependencies outside standard Ruby install and it's conky friendly! What else do you want  8)

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 16, 2016, 01:11:42 AM
While I am stoked beyond belief for your help hakerdefo and love the ruby solution... it is a far cry from what I have been used to and perhaps you might want to improve it

Here is what I see:

After working all day I have 36 new messages on Gmail and this is how the new Ruby shows it


  (http://www.zimagez.com/miniature/screenshot2016-11-1518-55-01.php)
(http://www.zimagez.com/zimage/screenshot2016-11-1518-55-01.php)

This is how the old shows it

(http://www.zimagez.com/miniature/screenshot2016-11-1518-53-49.png) (http://www.zimagez.com/zimage/screenshot2016-11-1518-53-49.php)

What would be need should be explained in the old script...

But, if no one can solve this the third and likable option is this bash option that simply shows how how may messages I have on gmail

mail-notify (make it executable)

#!/bin/bash

gmail_login="your_gmail_login_name" #login name
gmail_password="your_gmail_login_password" #pass

dane="$(wget --secure-protocol=TLSv1 --timeout=3 -t 1 -q -O - \
https://${gmail_login}:${gmail_password}@mail.google.com/mail/feed/atom \
--no-check-certificate | grep 'fullcount' \
| sed -e 's/.*<fullcount>//;s/<\/fullcount>.*//' 2>/dev/null)"

if [ -z "$dane" ]; then
echo "No connection"
else
echo "$dane message(s)"
fi


run it in conky like this:

${color #E2CF3B} ${execi 20 /home/vastone/mail-notify}

It looks like this:


  (http://www.zimagez.com/miniature/screenshot2016-11-1519-10-46.php)
(http://www.zimagez.com/zimage/screenshot2016-11-1519-10-46.php)
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 16, 2016, 09:42:23 PM
Okay, one more go at the python.

Gmail by default blocks all non-Google applications from accessing your account. There are two methods to make this script work with Gmail.

First Method (recommended) :-

Generate app password for this script and use that generated password in the script. More information here,
Sign in using App Passwords (https://support.google.com/accounts/answer/185833?hl=en)

Second Method :-

Change your Goolge account setting to allow non-Google apps access to your account. More information here,
Allowing less secure apps to access your account (https://support.google.com/accounts/answer/6010255?hl=en)

Open this script in your text-editor,


#!/usr/bin/env python2
import sys
import socket
import urllib
import errno
import os
import signal
from xml.dom import minidom
from functools import wraps

username = "your_gmail_username"
password = "your_gmail_password"
label    = str(sys.argv[1])
limit    = int(sys.argv[2])

class TimeoutError(Exception):
    pass

def timeout(seconds=10, error_message=os.strerror(errno.ETIME)):
    def decorator(func):
        def _handle_timeout(signum, frame):
            raise TimeoutError(error_message)

        @wraps(func)
        def wrapper(*args, **kwargs):
            signal.signal(signal.SIGALRM, _handle_timeout)
            signal.alarm(seconds)
            try:
                result = func(*args, **kwargs)
            finally:
                signal.alarm(0)
            return result

        return wrapper

    return decorator

GMAIL_URL = 'https://'+urllib.quote(username)+':'+urllib.quote(password)+'@mail.google.com/mail/feed/atom/'+urllib.quote(label)+'/'

def utfEncode(string):
    return unicode(string).encode('utf-8')

def printEmails(emails):
    for email in emails['emails']:
        print 'Sender: %s' %(utfEncode(email['address']))
        print 'Subject: %s' %utfEncode(email['title'])
        print 'Summary: %s\n' %utfEncode(email['summary'])

@timeout(10)
def getGmail(label,maxCount):
    url = GMAIL_URL
    try:
        xml = urllib.urlopen(url)
        dom = minidom.parse(xml)
    except:
        print '\nError: Could not get emails\n'
        exit()
    mail = []
    for node in dom.getElementsByTagName('entry'):
        if maxCount == 0:
          break
        author = node.getElementsByTagName('author')[0]
        mail.append({
            'title'  : node.getElementsByTagName('title')[0].firstChild.data,
            'summary': node.getElementsByTagName('summary')[0].firstChild.data,
            'sender' : author.getElementsByTagName('name')[0].firstChild.data,
            'address': author.getElementsByTagName('email')[0].firstChild.data
        })
        maxCount -= 1
    count = dom.getElementsByTagName('fullcount')[0].firstChild.data
    return {
        'emails'    : mail,
        'labelname' : label,
        'labelcount': count
    }

printEmails(getGmail(label,limit))



And change the following fields in the script appropriately,


username = "your_gmail_username"
password = "your_gmail_password"


Save the script somewhere in your $PATH with the name 'check-gmail.py' and give it execute permission,


chmod 755 /path/to/check-gmail.py


Open '.conkyrc' in your text-editor and add entry for 'check-gmail.py' like this,


execi 300 python2 /path/to/check-gmail.py INBOX 5


IMPORTANT:- Don't change 'python2' and 'INBOX' from the above snippet. The script won't work if you change either of them. Of-course you can change update interval from 300 seconds to some value of your choice. '5' in the above snippet determines the number of unread mail the script will display. You can change it from '5' to the value of your choice.

A scrot,

(https://cdn.scrot.moe/images/2016/11/16/screenshot_2016-11-17-023252.th.png) (https://scrot.moe/image/uQEy)

This worked for me as you can see from the scrot but your mileage may vary  ;)

This script won't work with python3. If you want to run it in the terminal, run it like this via python2,


python2 /path/to/check-gmail.py INBOX 5


Try it! What's the harm!

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 17, 2016, 12:15:30 AM
vastone@vsido:~$ python2 ./check-gmail.py INBOX 5

Error: Could not get emails


I cannot debug anything further... this is an epic script and I hope to get it working

Well done man...

V-Ger

Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 17, 2016, 07:11:22 AM
Quote from: VastOne on November 17, 2016, 12:15:30 AM
vastone@vsido:~$ python2 ./check-gmail.py INBOX 5

Error: Could not get emails


I cannot debug anything further... this is an epic script and I hope to get it working

Well done man...

V-Ger
That happened to me once or twice as well. Re-run the script and it returns the results. Retry it, chances are it'll work.
P.S. I again recommend you create and use a Google apps password with this script. Not only this script but every script-app that interacts with your account. You can create as many apps password as you wish.

Sign in using App Passwords (https://support.google.com/accounts/answer/185833?hl=en)

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 17, 2016, 12:50:50 PM
Still does not work for me.. restarted it several dozen times and even rebooted

Is there something new in the requirements from the import section?

import sys
import socket
import urllib
import errno
import os
import signal
from xml.dom import minidom
from functools import wraps
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 17, 2016, 05:42:26 PM
Quote from: VastOne on November 17, 2016, 12:50:50 PM
Still does not work for me.. restarted it several dozen times and even rebooted

Is there something new in the requirements from the import section?

import sys
import socket
import urllib
import errno
import os
import signal
from xml.dom import minidom
from functools import wraps

No, it's not a missing module problem. Had it been a module missing thingy python would have mentioned it in the error. Most likely Google is blocking the log-in attempt. If you are using your Google account password in the script then you'll have to change the account setting to allow non-Google apps access to your account. More information here,
Allowing less secure apps to access your account (https://support.google.com/accounts/answer/6010255?hl=en)

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 17, 2016, 07:01:58 PM
The original script (gmail-parser.py) that started the thread works fine on Debian stable. So we can assume that the one or more python modules packaged in Debian unstable is/are causing these scripts to fail on Debian unstable. One idea to try and resolve this problem is to use python modules from PyPI instead of Debian unstable.
First remove all python modules you have installed to meet the script dependencies. Like python-urllib3, python-requests, python-feedparser, python-openssl, python3-openssl, python3-requests, python3-urllib3, python3-feedparser etc. from the system. Next download the following package,

https://bootstrap.pypa.io/get-pip.py

cd to the download directory and run following commands in the exact order,


sudo python2 get-pip.py
sudo python3 get-pip.py


Then use pip to install modules required by a python script. For example if a python2 script requires 'feedparser' module, you can install it like this,

sudo pip2 install feedparser

And if a python3 script requires module 'feedparser', you can do,

sudo pip3 install feedparser

For more in-depth look at pip read this.

http://matthew-brett.github.io/pydagogue/installing_on_debian.html

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 18, 2016, 02:20:07 AM
Hi hakerdefo

It is definitely urllib/libssl that snap pointed out that is the issue. I am now on the machine that has sid running and just did an update and was told that libssl would be updated and that a bug is going to break python... nice.

I will try your new solutions but I already have my google acct setup as Access for less secure apps    and have verified with earlier scripts (ruby one) that it works as advertised.. It must be noted that I cannot change my google acct to any other setting. IOW, I do not have the option to go with different passwords for a more secure environment

I will get back with feedback
Title: Re: gmail-parser.py failure
Post by: VastOne on November 18, 2016, 03:01:47 AM
More weirdness...

I have disabled the check-gmail.py line from my conky script but yet the script just keeps on running... I do not see it in Task Manager or in top as a process running

The message

Error:Could not get emails

is explicit to the check-gmail.py script and when I delete the contents of that file and save it, only then does conky remove that message from my conky widget


Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 18, 2016, 06:04:13 AM
Quote from: VastOne on November 18, 2016, 02:20:07 AM
Hi hakerdefo

It is definitely urllib/libssl that snap pointed out that is the issue. I am now on the machine that has sid running and just did an update and was told that libssl would be updated and that a bug is going to break python... nice.

I will try your new solutions but I already have my google acct setup as Access for less secure apps and have verified with earlier scripts (ruby one) that it works as advertised.. It must be noted that I cannot change my google acct to any other setting. IOW, I do not have the option to go with different passwords for a more secure environment

I will get back with feedback
Hi there,
I would suggest you should first turn-off access to less secure apps here,

http://www.google.com/settings/security/lesssecureapps

Then whenever you need to provide your Google password to any app or script go the following page and create app password for that app/script,

https://security.google.com/settings/security/apppasswords

NOTE: Google will display the 16 character long app password only once when you create it.

You can always go to 'App passwords' page and remove an 'app password' whenever you want.
You can generate as many apps passwords as you wish so create-use a different app password for every application/script you use.

NOTE: You won't be able to create an App password if you have turned-on 'Access for less secure apps'.

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 18, 2016, 06:10:03 AM
Quote from: VastOne on November 18, 2016, 03:01:47 AM
More weirdness...

I have disabled the check-gmail.py line from my conky script but yet the script just keeps on running... I do not see it in Task Manager or in top as a process running

The message

Error:Could not get emails

is explicit to the check-gmail.py script and when I delete the contents of that file and save it, only then does conky remove that message from my conky widget
Strange indeed! As I've mentioned in my second last post you can use python modules from PyPI instead of Debian unstable whenever a script needs a module.

http://vsido.org/index.php?topic=1222.msg13817#msg13817

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 19, 2016, 12:29:58 AM
I cannot access to setup app passwords even after having less secure apps turned off for 24 hours

Google sucks with a lot of their bullshit and I am tired of it all

I appreciate all your help and would like to see this script (check-gmail.py) working but my patience has run out

Hopefully in the long run the original issue with python and urllib/libssl will work itself out and the original gmail-parser.py can be used again
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 19, 2016, 06:07:10 AM
Quote from: VastOne on November 19, 2016, 12:29:58 AM
I cannot access to setup app passwords even after having less secure apps turned off for 24 hours

Google sucks with a lot of their bullshit and I am tired of it all

I appreciate all your help and would like to see this script (check-gmail.py) working but my patience has run out

Hopefully in the long run the original issue with python and urllib/libssl will work itself out and the original gmail-parser.py can be used again

I think you will also have to enable 2-Step Verification in order to use 'App Passwords'. You can set-it-up here,

http://accounts.google.com/SmsAuthConfig

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: hakerdefo on November 19, 2016, 07:38:42 PM
Many a wayward wanderers are going to stumble in here via Google or DuckdDuckGo, looking for ways to get unread mails in conky, That's the reason why I'm putting this post in here, To guide those wayward wanderers to their destination  :D

Gmail unread email information in conky (http://vsido.org/index.php?topic=1225.0)

Cheers!!!
Title: Re: gmail-parser.py failure
Post by: VastOne on November 19, 2016, 10:38:28 PM
^^ HA!

Brilliant man...

;D  8)