Merge branch 'master' of gitorious@git.sugarlabs.org:question-support-api/mainline

Conflicts:
	quizdata/_urlproc.py

11 files changed, 313 insertions, 159 deletions

diff --git a/doc/TODO b/doc/TODO
new file mode 100644
index 0000000..69ba62f
--- /dev/null
+++ b/doc/TODO
@@ -0,0 +1,16 @@
+.. vim:filetype=rst:tw=79
+
+API Project TODO
+================
+
+    - API Project Wiki Page
+    - git Tutorial / Intro (resources)
+    - Touch base with Reporting API Team
+    - URL Design, API design
+    - Touch base with other teams, gather reqs
+    - Teacher authoring Activity?
+    - Establish milestones
+    - Post wiki page to mailing list
+    - Documentation practices -- project process stuff (reST, in-repo)
+
+
diff --git a/doc/milestones.rst b/doc/milestones.rst
new file mode 100644
index 0000000..799fbe3
--- /dev/null
+++ b/doc/milestones.rst
@@ -0,0 +1,31 @@
+Milestones
+==========
+
+    Initial Prototype Phase (1)
+        Simple implementation, data model not yet nailed down, focus on import
+        and utility to question *consumers*, like Activities. Initial formats
+        to include MoodleXML, GIFT.
+        
+    Rigorous Design Phase (2)
+        Nail down the data model, including developing the 'native' format,
+        probably using a sqlite file mechanism, or other database support, if
+        possible. Freeze the Question object (in terms of required properties,
+        etc.), finalize decisions about URLs for question aquisition.  Make
+        sure requirements of other projects *can* be met by the design at this
+        stage.
+
+    Full Implementation Phase (3)
+        Complete the implementation of the import formats, including the
+        'native' format.  Should be usable to other Activity developers at this
+        point (hopefully useful, before now, but all needs should be filled at
+        this point).  Export implementation should start now, along with
+        prototyping for an authoring activity. (Collaboration with the
+        reporting team *needs* to happen at this point, as the activity will
+        probably be combined with reporting tools.)
+
+    Activity Development Phase (4)
+        Complete the authoring/reporting activity for the teachers, allowing
+        export to file formats (and possibly *serving* the questions to other
+        XOs; requires support in activities (using the API) to support).
+
+
diff --git a/doc/quiz-api.rst b/doc/quiz-api.rst
index ba7e212..5ce1634 100644
--- a/doc/quiz-api.rst
+++ b/doc/quiz-api.rst
@@ -1,11 +1,8 @@
-.. vim:filetype=rst:tw=76:
-
-=================================
 Sugar Quiz API Preliminary Design
 =================================
 
 Motivation
-==========
+----------
 
 In the RIT class working on the Math4 projects, many proposed activities
 require a question database of some kind.  A common API or library for
@@ -21,7 +18,7 @@ grouping, difficulty, and subject matter that would be part of the base
 system.
 
 Envisioned Usage
-================
+----------------
 
 Consider a simple flash-card-like activity.  It presents a question from a
 list of questions, allows the student to select an answer from the provided
@@ -30,7 +27,7 @@ answer is revealed and the student it told whether their answer is correct.
 If the question has an explanation of the correct answer, the flash-card
 activity will show the explanation of the correct answer.  (Note that this
 is just a simple usage example, the interaction design of a drilling
-activity could be markedly different.)
+activity could be very different.)
 
 The flash-card activity would use this proposed Quiz API for the following:
 
@@ -41,9 +38,6 @@ The flash-card activity would use this proposed Quiz API for the following:
 
     - Determining whether the student has entered a correct answer.
 
-    - Rendering the question to a simple widget/canvas. (i.e. pass the
-      library a GtkCanvas or similar and tell it to display the question)
-
 To start with, the library would simply be a time-saving tool for developers
 needing similar functionality, but as the XS (School Server) becomes more
 fully developed the library should integrate the functions provided by the
@@ -52,23 +46,22 @@ study so the students can drill material using any tool they prefer, while
 still reporting progress to the instructor using the XS services.
 
 Proposed API
-============
+------------
 
-The Quiz API would be a python library, to act mostly as glue between
-various file formats (and local or remote resources) for question data and
-the Gtk graphical environment, providing a simple way to consistently
-present and layout questions.
+The Quiz API would be a python library, to act mostly as glue between various
+file formats (and local or remote resources) for question data and the client
+Activity.
 
     :quizdata.open(uri, [cache=False]):
         Opens a URI, returning a list of quizdata.Question instances.  A
         standard method of filtering question data based on parameters
         should be specified.  Examples of URIs that might be used::
 
-            http://xs-server/math4class/current_topic?level=4&difficulty=hard&format=moodle
+            http://xs-server/math4class/current_topic?format=gift
 
-            file:///var/lib/mathquestions/quiz1?level=4&difficulty=hard&format=xml
+            file:///var/lib/mathquestions/quiz1?format=moodlexml
 
-            xmpp://teacheraccount@xs.server/classname?difficulty=hard&level=4
+            xmpp://teacheraccount@xs.server/classname?format=gift&difficulty=hard&level=4
 
         The cache parameter would locally save the retrieved questions to a
         persistent storage so requests from the same URI (with cache=True)
@@ -79,8 +72,9 @@ present and layout questions.
             - The question text
             - The style of answer (incl. multiple-choice, numeric, free
               response, etc.)
-            - The correct answer (or if the question is subjective, that
-              there *is* no correct answer).
+            - The correct answer (or if the question is subjective, that there
+              *is* no correct answer). This includes answers for which the
+              student may receive partial credit.
             - Question difficulty
             - Grade level
             - Tags (for free-form grouping by topic, course, instructor,
@@ -88,68 +82,45 @@ present and layout questions.
 
         The question text and answers should support at least minimal
         markup, like that supported by pango, in addition to markup
-        rendering with MathML/LaTeX syntax.
+        rendering with MathML/LaTeX syntax. (This requires a markup_type
+        parameter of some kind.)
 
         .. note:: 
             The attributes listed above will should grow standardized names
             and be documented as part of the interface to the Question
-            class, to allow for fine-grained for activity controlled
-            rendering of the Question, if the simple show() call is not
-            appropriate.
-
-        :Question.show(surface, x, y, [width=-1, [height=-1]]):
-            Draw the question to the drawing surface at coordinates (x,y)
-            limited to the optionally specified width/height.
-
-            This also should set up the appropriate input widgets for the
-            type of question (multiple-choice/free-response) and handle the
-            vents for those widgets.
-
-        :Question.response:
-            The answer the student has currently selected, or None if no
-            answer has been entered.
-
-        :Question.submitted:
-            True if the student has submitted an answer for the Question,
-            False otherwise.
+            class.
 
-        :Question.answered():
+        :Question.answered:
             Returns True if the student has provided an answer for the
             Question.
 
-        :Question.correct():
+        :Question.submitted:
+            Returns True if the student has submitted an answer for the
+            Question. XXX: Useful to make a distinction between answered and
+            submitted?
+
+        :Question.correct:
             Returns True if the currently selected answer is correct for the
-            Question.
+            Question. XXX: What to do about partial credit? Value between 0 and
+            1?
 
-        :Question.clear():
-            Removes the widgets and drawings that show() set up, preparing
-            the surface to receive the next question or other widgets.
+        :Question.answer:
+            Returns the answer the student has currently selected, or None
+            if no answer has been entered.
 
 
 Implementation Issues
-======================
-
-The implementation of this (deceptively simple) library will take some
-effort, in that it will be closely tied to the windowing/graphical toolkit,
-PyGtk/Cairo/Pango rather directly, due to the high level of abstraction.
-Additionally the URI lookup and question filtering based on parameters will
-be necessary, as will interpreter the various format parsers necessary to
-build the Question objects.
-
-For MathML support, the GtkMathView widget will need to be available, so a
-certain amount of effort may be involved in packaging the library in a
-simple way for activity developers.
-
-Next Steps
-==========
-
-Firstly, this API is being proposed and posted to the Math4 mailing list for
-feedback and changes before any commitments to this interface is decided.
-For any activity developers who are currently working on an activity that
-could take advantage of such a system, or who have written similar
-functionality in an activity, your input on usage and the naturalness of the
-API.
-
-Secondly, anyone who is interested in doing work on this library or using
-the library in their activity should chime in, along with the expected usage
-or how you can contribute.
+----------------------
+
+The URI lookup and question filtering based on parameters will be necessary.
+Further design and discussion is required.
+
+Parsing the various format parsers necessary to build the Question objects.
+
+For markup support, some simple way to give the client activities and easy way
+to display it would be desirable. (Restrict the list of markup formats? Require
+supplying plain text in addition?)
+
+.. note:: 
+    File URI handling right now is weird, as the python urlparse stuff doesn't
+    parse file:// scheme URI's with query parameters.
diff --git a/quizdata/__init__.py b/quizdata/__init__.py
index 529d10c..2bddc6e 100644
--- a/quizdata/__init__.py
+++ b/quizdata/__init__.py
@@ -3,5 +3,9 @@
 '''
 from __future__ import absolute_import
 
+import re
+
+from peak.rules import when, abstract, around
+
 from ._question import Question
 from ._urlproc import open
diff --git a/quizdata/_format_gift.py b/quizdata/_format_gift.py
index 1321268..f84da79 100644
--- a/quizdata/_format_gift.py
+++ b/quizdata/_format_gift.py
@@ -81,9 +81,17 @@ question.ignore(comment)
 questions = delimitedList(question, delim=OneOrMore(NL))("questions") + ZeroOrMore(NL) + StringEnd()
 questions.ignore(comment)
 
-def parse_only(text):
-    return questions.parseText(text)
-
-def parse(text):
-    raise NotImplementedError()
+def _search_file(parser, stream):
+    return parser.searchString(stream.read())
+
+def parse(stream, params):
+    parsed = _search_file(question, stream)
+    ret = []
+    for q in parsed:
+        ret.append(_question_maker(q))
     
+    return ret
+
+def _question_maker(q):
+    #XXX: NotImplemented
+    return q
diff --git a/quizdata/_question.py b/quizdata/_question.py
deleted file mode 100644
index 10f7c87..0000000
--- a/quizdata/_question.py
+++ /dev/null
@@ -1,63 +0,0 @@
-'''
-    quizdata.Question implementation and support functions
-'''
-from peak.util.symbols import Symbol
-
-TRUE_FALSE = Symbol('TRUE_FALSE', __name__)
-MULTI_CHOICE = Symbol('MULTI_CHOICE', __name__)
-SHORT_ANSWER = Symbol('SHORT_ANSWER', __name__)
-MATCHING = Symbol('MATCHING', __name__)
-MISSING_WORD = Symbol('MISSING_WORD', __name__)
-NUMERICAL = Symbol('NUMERICAL', __name__)
-
-_FRIENDLY_NAMES = {
-        TRUE_FALSE : 'true/false',
-        MULTI_CHOICE : 'multiple choice',
-        SHORT_ANSWER : 'short answer',
-        MATCHING : 'matching', 
-        MISSING_WORD : 'missing word',
-        NUMERICAL : 'numerical',
-    }
-
-class Question(object):
-    def __init__(self, text, type, answers=None, title=None, category=None,
-                diffculty=None, level=None, tags=None):
-        self.text = text
-
-        if type not in [ TRUE_FALSE, MULTI_CHOICE, SHORT_ANSWER, MATCHING,
-                         MISSING_WORD, NUMERICAL ]:
-                raise Exception("Type must be one of the defined question types")
-        self.type = type
-
-        if answers is None and self.type != TRUE_FALSE:
-            raise Exception("Answers are required for questions of type %d" % self.type)
-
-        if title is None:
-            self.title = self.text
-
-        self.category = category
-        self.difficulty = difficulty
-        self.level = level
-        if tags is None:
-            self.tags = []
-
-
-        self.submitted = False
-        self.response = None
-
-    def show(self, surface, x, y, width=-1, height=-1):
-        raise NotImplementedError()
-
-    def clear(self):
-        raise NotImplementedError()
-
-    def answered(self):
-        return bool(self.response)
-
-    def correct(self):
-        raise NotImplementedError()
-
-    def __repr__(self):
-        return "<%s Question: %s>" % (_FRIENDLY_NAMES[self.type].title(), 
-                                      self.title)
-
diff --git a/quizdata/_urlproc.py b/quizdata/_urlproc.py
index c877189..7eea086 100644
--- a/quizdata/_urlproc.py
+++ b/quizdata/_urlproc.py
@@ -1,12 +1,37 @@
 '''
     Module implementing the opening of a quizdata resource via URL.
 '''
-from urllib import urlopen
+import urllib2
+from urlparse import urlparse
+from cgi import parse_qs
+from .formats import FORMATS
+
+QuestionOpener = urllib2.build_opener()
 
 def open(url, cached=False):
-    if (not url.startswith("http") or
-        not url.startswith("ftp") or
+    if (not url.startswith("http") and
         not url.startswith("file")):
         raise NotImplementedError()
 
-    raise NotImplementedError()
+    parsed_url = urlparse(url)
+    params = parse_qs(parsed_url.query)
+    if not params:
+        alt_parse = url.rsplit('?',1)
+        params = parse_qs(alt_parse[-1])
+        url = alt_parse[0]
+
+    params = _delist(params)
+
+    if 'format' not in params:
+        print parsed_url
+        print params
+        raise Exception("format parameter is required")
+
+    return FORMATS[params['format']](
+            QuestionOpener.open(url), 
+            params)
+
+def _delist(d):
+    for k, v in d.items():
+        d[k] = v[0]
+    return d
diff --git a/quizdata/question.py b/quizdata/question.py
new file mode 100644
index 0000000..7c20a28
--- /dev/null
+++ b/quizdata/question.py
@@ -0,0 +1,112 @@
+'''
+    quizdata.Question implementation and support functions
+'''
+class Question(object):
+    '''
+        Base question type, abstract.
+    '''
+    def __init__(self, *args, **kwargs):
+        '''
+            Takes all a the data elements for the construction of the question
+            object.
+
+            :Parameters:
+                title : str or unicode
+                    the title of the question. optional.  if not supplied, set
+                    to None.
+                markup_type : type or factory
+                    the type used to coerce or adapt the raw 'markup' supplied
+                    by the backend to the correct format for use by the
+                    application. if not supplied a default of 'str' is used.
+                text : str or unicode
+                    the text of the question, assigned the result of
+                    markup_type(text)
+                tags : list of str  or unicode
+                    a list of textual  tags used to categorize or filter the
+                    question.
+        '''
+        if 'title' in kwargs:
+            self.title = kwargs['title']
+        else:
+            self.title = None
+
+        if 'markup_type' in kwargs:
+            self.markup_type = kwargs['markup_type']
+        else:
+            self.markup_type = str
+
+        if 'text' in kwargs:
+            self.title = self.markup_type(kwargs['text'])
+        else:
+            raise Exception("Questions must have text!")
+
+        if 'tags' in kwargs:
+            self.tags = kwargs['tags']
+        else:
+            self.tags = []
+
+    def __repr__(self):
+        if self.title:
+            txt = self.title
+        else:
+            txt = self.text
+        return "<%s %r>" % (self.__class__.__name__, txt)
+
+
+class MultipleChoiceQuestion(Question):
+    def __init__(self, *args, **kwargs):
+        super(MultipleChoiceQuestion, self).__init__(*args, **kwargs)
+        if 'answers' in kwargs:
+            self.answers = []
+            for ans in kwargs['answers']:
+                self.answers.append(self.markup_type(ans))
+        else:
+            raise Exception("MultipleChoiceQuestion requires answers!")
+
+        if 'correct' in kwargs:
+            self.correct = self.markup_type(kwargs['correct']) 
+        else:
+            raise Exception("MultipleChoiceQuestion requires a correct answer!")
+
+
+class MissingWordQuestion(MultipleChoiceQuestion):
+    def __init__(self, *args, **kwargs):
+        super(MissingWordQuestion, self).__init__(*args, **kwargs)
+        if 'tail_text' in kwargs:
+            self.tail_text = self.markup_type(kwargs['tail_text'])
+        else:
+            raise Exception("MissingWordQuestion requires a tail_text!")
+
+
+class TrueFalseQuestion(MultipleChoiceQuestion):
+    def __init__(self, *args, **kwargs):
+        kwargs['answers'] = ['True', 'False']
+        super(TrueFalseQuestion, self).__init__(*args, **kwargs)
+
+
+class ShortAnswerQuestion(Question):
+    def __init__(self, *args, **kwargs):
+        super(ShortAnswerQuestion, self).__init__(*args, **kwargs)
+        if 'correct' in kwargs:
+            self.correct = self.markup_type(kwargs['correct'])
+        else:
+            raise Exception("ShortAnswerQuestion requires a correct answer!")
+
+
+class NumericalQuestion(Question):
+    def __init__(self, *args, **kwargs):
+        super(NumericalQuestion, self).__init__(self, *args, **kwargs)
+        raise NotImplementedError("Not implemented yet!")
+
+
+class MatchingQuestion(Question):
+    def __init__(self, *args, **kwargs):
+        super(MatchingQuestion, self).__init__(*args, **kwargs)
+        if 'answers' in kwargs:
+            ans = []
+            for a1, a2 in kwargs['answers']:
+                ans.append((self.markup_type(a1), self.markup_type(a2)))
+            self.answers = ans
+        else:
+            raise Exception("MatchingQuestion must have answers!")
+
diff --git a/quizdata/text.py b/quizdata/text.py
new file mode 100644
index 0000000..2ac6d7a
--- /dev/null
+++ b/quizdata/text.py
@@ -0,0 +1,34 @@
+'''
+    markup / media -- conversion / coercion
+'''
+class htmlstr(str):
+    pass
+
+class uhtmlstr(unicode):
+    pass
+
+@abstract()
+def plain_text(s):
+    ''' Returns the text passed as a markup free / plain text string. '''
+
+@abstract()
+def html_text(s):
+    ''' Returns the text passed as an html marked-up text string. '''
+
+@when(html_text, (str, unicode))
+def txt2html(s):
+    return  ''.join(['<p>', s, '</p>' ])
+
+@when(html_text, (htmlstr, uhtmlstr))
+def html2html(s):
+    return s
+
+@when(plain_text, (str, unicode))
+def txt2txt(s):
+    return s
+
+@when(plain_text, (htmlstr, uhtmlstr))
+def html2txt(s):
+    return re.sub(r'<[^>]*?>','',s)
+
+
diff --git a/tests/test_gift_parse.py b/tests/test_gift_parse.py
index 2e21b81..366decb 100755
--- a/tests/test_gift_parse.py
+++ b/tests/test_gift_parse.py
@@ -12,17 +12,17 @@ from quizdata import _format_gift
 def test_example():
     full_example = open(path.join(base_path, 'tests', "examples.txt"))
 
-    print _format_gift.questions.parseFile(full_example)
+    print _format_gift.question.searchString(full_example.read())
 
 def test_text_nlnl():
-    print(_format_gift.text.parseString("""
+    print(_format_gift.text.searchString("""
     some text
     with newlines
     that should stay together
     """))
 
 def test_text_nlnl2():
-    print(_format_gift.text.parseString("""
+    print(_format_gift.text.searchString("""
     some text
     with newlines
 
@@ -30,26 +30,26 @@ def test_text_nlnl2():
     pieces"""))
 
 def test_text():
-    print(_format_gift.text.parseString(
+    print(_format_gift.text.searchString(
     "this is ? some ! text that should count as ' a single \"bit\" \
      of text to the GIFT parser."))
 
 def test_simple_tf():
-    print(_format_gift.question.parseString("4 is an even number{TRUE}\n"))
+    print(_format_gift.question.searchString("4 is an even number{TRUE}\n"))
 
 def test_simple_multi():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     What is the capital of France?{=Paris ~London ~Guam ~Tomato}
 
     """))
 
 def test_title2():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     ::Capital of France::The capital of France is Paris.{T}
     """))
     
 def test_title():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     ::Capital of France
     ::What is the capital of France? {
         =Paris
@@ -62,7 +62,7 @@ def test_title():
     """))
 
 def test_explain():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     What is an integer?{
         ~A whole number     # Whole numbers are only positive
         ~The natural numbers plus their negations   # Mostly true, may or may not include zero.
@@ -71,7 +71,7 @@ def test_explain():
     }"""))
 
 def test_matching():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     Match the countries with their capitals. {
         = Italy -> Rome
         = USA -> Washington D.C.
@@ -81,7 +81,7 @@ def test_matching():
 
 
 def test_questions():
-    print(_format_gift.questions.parseString("""
+    print(_format_gift.questions.searchString("""
 Matching Question. {
     =subquestion1 -> subanswer1
     =subquestion2 -> subanswer2
@@ -97,19 +97,19 @@ Match the following countries with their corresponding capitals. {
 """))
 
 def test_missing_word():
-    print(_format_gift.questions.parseString("""
+    print(_format_gift.questions.searchString("""
     The {=farmer ~hare} was the protagonist.
 
     The mother was a sympathetic character.{T}
     """))
 
 def test_format():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     [markdown]Who *invaded* France in 1882?{=Nobody ~The Polish ~The Greeks ~Everybody}
     """))
 
 def test_format2():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     [markdown]
     ::Invasion of France
     ::Who *invaded* France in 1882? {
@@ -121,7 +121,7 @@ def test_format2():
     """))
 
 def test_numerical():
-    print(_format_gift.questions.parseString("""
+    print(_format_gift.questions.searchString("""
     What is the value of pi (to 3 decimal places)? {#3.141..3.142}.
 
     [reST]Why does parsing this format *have* to be so **hard**? {
@@ -130,7 +130,7 @@ def test_numerical():
     """))
 
 def test_numerical2():
-    print(_format_gift.question.parseString("""
+    print(_format_gift.question.searchString("""
     What is the ratio of the circumference of a circle to its diameter to 3 decimal places? 
     {#3.1415:0.0005}"""))
 
diff --git a/tests/test_machinery.py b/tests/test_machinery.py
new file mode 100644
index 0000000..52ca36b
--- /dev/null
+++ b/tests/test_machinery.py
@@ -0,0 +1,16 @@
+import sys
+from os import path
+
+base_path = path.abspath(path.join(path.dirname(path.abspath(__file__)),'..'))
+sys.path.append(base_path)
+
+import quizdata
+
+def test_open_gift_file():
+    url = "file://%s?format=gift" % path.join(base_path, 'tests', 'examples.txt')
+    print url
+    questions = quizdata.open(url)
+    print questions
+
+if __name__=='__main__':
+    test_open_gift_file()