Resources I Collected And Referenced To Build Perplexity
At some point I may organize this better, but for now it serves as a kind of stream of consciousness record of all the papers and sites I found as I researched Perplexity and tried to get my head around it.
Parsers and Toolkit Options I Evaluated
- Boxer parser: http://www.let.rug.nl/bos/pubs/Bos2015NoDaLiDa.pdf
- NLTK
- Cornell Semantic Parsing Framework: https://github.com/lil-lab/spf
- Unclear if this has implementation of english
- Has a Copyleft license.
- Stanford CoreNLP
- Apache OpenNLP
- Spacy
- GenSim
- TextBlob
- Graphene is an information extraction pipeline which extracts Knowledge Graphs from texts: http://lambda3.org/Graphene/
- Sempre parser: https://github.com/percyliang/sempre
- Supports lambda DCS logical forms
- Appears to be shallow: https://github.com/pcyin/tranX
- Semantic Graph parser: https://github.com/kkalouli/GKR_semantic_parser
- Parser for Natural Semantic Metalanguage: https://github.com/Yakushima/NSM-DALIA/tree/master/nsm-dalia-0.9
- Statistical: https://github.com/janmbuys/DeepDeepParser
- Type driven semantic parser: https://github.com/kaayy/TISP
- Has a demo
- Does not convert to logical form
- CCG Semantic Parsing library for Scala: https://github.com/Workday/upshot-montague
- Conceptual Semantics in NLP toolkit
- SippyCup: https://github.com/wcmac/sippycup
- Understanding verbs: http://verbnet-semantic-parser.appspot.com/
The English Resource Grammar
- Overview http://www.delph-in.net/erg/
- Format of MRS: http://moin.delph-in.net/MrsRfc
- Docs: http://moin.delph-in.net/ErgSemantics
- Fingerprint search: http://moin.delph-in.net/ErgSemantics
- Good overview and background in some chapters here:
- Message Board: https://delphinqa.ling.washington.edu/
- Online ERG parser: http://erg.delph-in.net/logon
- Python Interface: https://github.com/delph-in/pydelphin
- The ACE Parser: http://sweaglesw.org/linguistics/ace/
- These guys show how a lot of the phrasing works: http://www.cs.jhu.edu/~xuchen/paper/Yao2010Master.pdf
Linguistics
Grammars
Montague Grammar
- The Proper Treatment of Quanitifiers, Montague 1973
- One useful interpretation of it
- Good description of the challenges of Montague Grammar:
- Lambda operators in Prolog: https://www.complang.tuwien.ac.at/ulrich/Prolog-inedit/ISO-Hiord
- http://people.umass.edu/partee/docs/MontagueGrammarElsevier.PDF
- Another: http://www-personal.umich.edu/~akao/NLP_Paper.htm
- Another: https://plato.stanford.edu/entries/montague-semantics/
(Neo) Davidsonian Grammar
- Davidson, Donald. 1967. The logical form of action sentences
- Good background on neodavidson: http://verbs.colorado.edu/~mpalmer/Ling7800/ParsonsMP.pdf
- Very good summary that motivates why you need the event argument: https://user.phil-fak.uni-duesseldorf.de/~filip/Davidson.67.pdf
- Another great summary: https://people.umass.edu/partee/NZ_2006/NZ20TenseAspectEvents.pdf
Events in Linguistics
- Very interesting background: http://www.linguist.org/files/History-of-Events.pdf
- REALLY useful: http://www.sfu.ca/~dearmond/322/LF-Sentence.pdf
- This seems to explain where events are used for adjectives: https://core.ac.uk/download/pdf/14509892.pdf
- OK, this really gets to the root of why adjectives have them: http://semantics.uchicago.edu/kennedy/docs/routledge.pdf
Generalized Quantifiers
- Basic description: http://ling-blogs.bu.edu/lx502s13/files/2013/04/LX502_RestrictedQuantifiers.pdf
- Good Overview
- Another Good Overview
- Another: https://sites.google.com/site/heathersusanburnett/home/teaching/egg2012-quantification
- This (surprisingly) has a great description of Quantifiers in logic and NL: https://www.encyclopedia.com/humanities/encyclopedias-almanacs-transcripts-and-maps/quantifiers-natural-language
- This describes how restriction and scope works: http://ling-blogs.bu.edu/lx502s13/files/2013/04/LX502_RestrictedQuantifiers.pdf
- The end of this describes a lot of good stuff! http://sharif.edu/~sani/courses/nlp/lec6.pdf
- Converting “every” “most” etc to logic. “most” is not first order! https://people.umass.edu/partee/409/Logic_and_Natural_Language.pdf
Morphosemantic Properties
- Morphosemantic Properties:
- Tense - location in time
- Aspect - fabric of time: a single block of time, continuous flow of time, or repetitive occurrence
- Mood - degree of necessity, obligation, probability, ability
- From: https://en.wikipedia.org/wiki/Grammatical_mood
- Some examples of moods are indicative, interrogative, imperative, subjunctive, injunctive, optative, and potential.
Using Logic With Natural Language
Composability
Natural Language Generation
- Natural Language Generation
- https://pypi.org/project/simplenlg/
Natural Language Understanding/Processing
Good example of why we need deep (i.e. full understanding) vs. shallow: https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1162/handouts/Computational-Semantics.pdf
- Also a good summary of existing approaches
NLP Background:
- https://www.tutorialspoint.com/natural_language_processing/index.htm
- https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3168328/
- https://www.kdnuggets.com/2019/02/deep-learning-nlp-rnn-cnn.html
- https://www.analyticsvidhya.com/blog/2019/03/pretrained-models-get-started-nlp/
- https://towardsdatascience.com/natural-language-processing-from-basics-to-using-rnn-and-lstm-ef6779e4ae66
Logic
- Predicate Logic Overview
- Situational Calculus: representation and reasoning about time, fluents, and the use of logic for representing knowledge in a computer.[12][13]
- Non-monatonic logic (https://en.wikipedia.org/wiki/Non-monotonic_logic): system associates facts asserted with the rules and facts used to justify them and as those facts change updates the dependent knowledge as well.
Conceptual Graphs
- Initial Definition: http://www.jfsowa.com/pubs/cg1976.pdf
- Intro: http://homepages.gold.ac.uk/polovina/publications/46040001-SP-Intro-to-CG.pdf
- Deeper intro: http://www.jfsowa.com/cg/cg_hbook.pdf
- http://www.cs.fsu.edu/~lacher/courses/CAP5605/lectures/SixthEdChap7.ppt
- More info: https://www.deepdyve.com/lp/oxford-university-press/conceptual-graphs-and-first-order-logic-zBiNtoJHSJ
- Representing Conceptual graphs in prolog: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.48.6204&rep=rep1&type=pdf
Ontologies
- How to build an ontology: https://protege.stanford.edu/publications/ontology_development/ontology101-noy-mcguinness.html
- OWL
- Members of the OWL family have model theoretic formal semantics, and so have strong logical foundations.
- Has wide support including Prolog
- Critiques of OWL
- OPM seems to be a good basic model: https://en.wikipedia.org/wiki/Object_Process_Methodology
Naïve Physics (what most humans have as a model of physics)
- Naïve Physics Manifesto: http://ontology.buffalo.edu/smith/articles/naivephysics.html
- Naïve Physics Perplex: https://pdfs.semanticscholar.org/0cb8/9b2135496e5ab4d8fef97866a6d9c88b1ba4.pdf
- Egg Cracking: http://www-formal.stanford.edu/leora/commonsense/eggcracking.html
- Formalization of motion: http://www.qrg.northwestern.edu/papers/Files/QRG_Dist_Files/QRG_1980/2_122_Forbus_1980_Spatial_Qualitative_Aspects.pdf
Common Sense Reasoning/Scaled Data
- Keywords: Knowledge representation and reasoning (KR², KR&R)
- Cyc (referenced often, hard to find real data)
- Parsing limited english into logic statements so we can train a system: https://github.com/tiantiangao7/kalm/blob/master/docs/KALM_ODBASE18.pdf
- Uses Framenet and BebelNet to get stuff done
- https://en.wikipedia.org/wiki/Semantic_reasoner
- FrameNet: https://framenet.icsi.berkeley.edu/fndrupal/framenet_search
- WordNet: WordNet is the most commonly used computational lexicon of English for word sense disambiguation (WSD),
- A common use of WordNet is to determine the similarity between words.
- Just using synnets for finding synonyms is not enough
- https://www.nltk.org/howto/wordnet.html
- Conceptnet
- Using ConceptNet to Teach Common Sense to an Automated Theorem Prover: http://arcade2019.net/pdfs/6.pdf
- BabelNet: Take wordnet and apply to wikipedia
- Gellish English dictionary
- The Gellish English Dictionary-Taxonomy is an example of an open-source “smart” electronic dictionary, in which concepts are arranged in a subtype-supertype hierarchy, thus forming a taxonomy.
- Putting Pieces Together: Combining FrameNet, VerbNet and WordNet for Robust Semantic Parsing
Converting English into Logic
- Chat 80: https://era.ed.ac.uk/bitstream/handle/1842/6646/Pereira1982.pdf
- See how well/natural Attempto is in turning english into Prolog logic:
- Natural Semantic metalanguage: https://intranet.secure.griffith.edu.au/schools-departments/natural-semantic-metalanguage
- Deep Semantic Interpretations Of Legal Texts using DCG and FOL logic: https://www.cs.rutgers.edu/~mccarty/research/icail07-acm.pdf
- Prolog and NLU: https://www.researchgate.net/publication/2827995_Natural-Language_Interpretation_in_Prolog
- Prolog and NLU: http://www.mtome.com/Publications/PNLA/prolog-digital.pdf
- https://www.researchgate.net/publication/2262147_Masquesql–_An_Efficient_and_Portable_Natural_Language_Query_Interface_for_Relational_Databases
- What to do and how to do it: Translating Natural Language Directives into Temporal and Dynamic Logic: https://hrilab.tufts.edu/publications/dzifcaketal09icra.pdf
- Commanding a robot: https://homes.cs.washington.edu/~lsz/papers/mhzf-iser12.pdf
Micro Worlds
- Zoo World: https://www.ida.liu.se/ext/caisor/pm-archive/krf/021/PM-krf-021.pdf
- Logic Modeling Workshop: https://www.ida.liu.se/ext/etai/lmw/
- Processing language defining the world: https://www.ijcai.org/Proceedings/81-1/Papers/001.pdf
- SOME BASIC MECHANISMS FOR COMMON SENSE REASONING ABOUT STORIES ENVIRONMENTS
Creating Plots/Narratives
- Basic model of plots, how to generate a good story: http://www.erasmatazz.com/library/the-journal-of-computer/jcgd-volume-7/flawed-methods-for-interact.html
- Survey from 2013: https://nil.cs.uno.edu/publications/papers/roberts2008survey.pdf
- Great pointers to lots of data:
- Erasmatron: http://ifwiki.org/index.php/Erasmatron
- Scheherazade IF (Boyang Li and Mark O. Riedl): https://www.cc.gatech.edu/~riedl/pubs/aaai15.pdf
- Build simple narratives from plot graphs:
- Generates partially ordered plots using mechanical turk
- Dilemma Based narrative: http://www.aaai.org/Papers/AIIDE/2007/AIIDE07-001.pdf
- Good for generating infinite stories, limited vocab and storytelling text
- User is coerced into doing things to make a dilemma happen
- On being passed a dilemma the planner finds all plans to achieve this dilemma given the current storyworld state and background knowledge.
- Toward Automated Quest Generation in Text-Adventure Games: https://deepai.org/publication/toward-automated-quest-generation-in-text-adventure-games
- Improvisational Story Elements: https://nips2017creativity.github.io/doc/Improvisational_Agents.pdf
- Tale-spin: https://www.cs.utah.edu/nlp/papers/talespin-ijcai77.pdf
- Data-Driven Personalized Drama Management (Hong Yu and Mark O. Riedl): https://www.cc.gatech.edu/~riedl/pubs/aiide13.pdf
- Façade drama manager: https://pdfs.semanticscholar.org/a5d2/af2a518e2c74761bdc3d976657ac48c9d2f8.pdf
- UNIVERSE: https://nil.cs.uno.edu/publications/papers/lebowitz1985story.pdf
- Comme il faut: https://nil.cs.uno.edu/publications/papers/mccoy2014cif.pdf
- Emotional behavior that is generated
Dialog
- ChatScript: https://sourceforge.net/projects/chatscript/
- The Second Conversational Intelligence Challenge (ConvAI2): https://arxiv.org/pdf/1902.00098.pdf
- Learning to Speak and Act in a Fantasy Text Adventure Game: https://arxiv.org/pdf/1903.03094.pdf
- https://parl.ai/projects/light/
- Personalizing Dialogue Agents: I have a dog, do you have pets too?: https://arxiv.org/pdf/1801.07243.pdf
- POMDP-Based Statistical Spoken Dialog Systems: A Review: http://cs.brown.edu/courses/csci2951-k/papers/young13.pdf
- Bruce Wilcox Chatbot:https://en.wikipedia.org/wiki/Bruce_Wilcox
- Breakdown of games with dialog: https://emshort.blog/2018/09/11/mailbag-deep-conversation/
- Conversation modeling: https://emshort.blog/2019/04/16/mailbag/#more-39174
- Character Engine: https://www.spiritai.com/character-engine/
- Scripts: A script is a structured representation describing a stereotyped sequence of events in a particular context. The script was originally designed by Schank and his research group 1Schank and Abelson 1977)
Interactive Fiction
- https://emshort.blog/
- Discussion of depth of parsing in IF from Emily Short: https://emshort.blog/2010/06/07/so-do-we-need-this-parser-thing-anyway/
- List of IF authoring systems: http://www.ifwiki.org/index.php/Category:Authoring_system
The Inform 7 Platform
- Introduction
- Background on Implementation
Games and Systems
- Chat 80
- http://aidungeon.io
- “Mastering the Dungeon” a mechanical turk AI text adventure
- https://arxiv.org/pdf/1711.07950v2.pdf
- “The world is represented as a set of objects, along with directed typed edges indicating the relations between them. The set of possible actions of an agent are then defined as updates to the graph structure. “
- Bot Colony: https://store.steampowered.com/app/263040/Bot_Colony/
- https://talktotransformer.com
- https://tccoxon.itch.io/dreamer
- Blockchain adventure where you pay to add to the story