Resources I Collected And Referenced To Build Perplexity
At some point I may organize this better, but for now it serves as a kind of stream of consciousness record of all the papers and sites I found as I researched Perplexity and tried to get my head around it.
Parsers and Toolkit Options I Evaluated
- Boxer parser: http://www.let.rug.nl/bos/pubs/Bos2015NoDaLiDa.pdf
- NLTK
- Cornell Semantic Parsing Framework: https://github.com/lil-lab/spf
- Unclear if this has implementation of english
- Has a Copyleft license.
- Stanford CoreNLP
- Apache OpenNLP
- Spacy
- GenSim
- TextBlob
- Graphene is an information extraction pipeline which extracts Knowledge Graphs from texts: http://lambda3.org/Graphene/
- Sempre parser: https://github.com/percyliang/sempre
- Supports lambda DCS logical forms
- Appears to be shallow: https://github.com/pcyin/tranX
- Semantic Graph parser: https://github.com/kkalouli/GKR_semantic_parser
- Parser for Natural Semantic Metalanguage: https://github.com/Yakushima/NSM-DALIA/tree/master/nsm-dalia-0.9
- Statistical: https://github.com/janmbuys/DeepDeepParser
- Type driven semantic parser: https://github.com/kaayy/TISP
- Has a demo
- Does not convert to logical form
- CCG Semantic Parsing library for Scala: https://github.com/Workday/upshot-montague
- Conceptual Semantics in NLP toolkit
- SippyCup: https://github.com/wcmac/sippycup
- Understanding verbs: http://verbnet-semantic-parser.appspot.com/
The English Resource Grammar
- Overview http://www.delph-in.net/erg/
- Format of MRS: http://moin.delph-in.net/MrsRfc
- Docs: http://moin.delph-in.net/ErgSemantics
- Wiki: https://github.com/delph-in/docs
- Fingerprint search: http://moin.delph-in.net/ErgSemantics
- Good overview and background in some chapters here:
- Message Board: https://delphinqa.ling.washington.edu/
- Online ERG parser: http://erg.delph-in.net/logon
- Python Interface: https://github.com/delph-in/pydelphin
- The ACE Parser: http://sweaglesw.org/linguistics/ace/
- These guys show how a lot of the phrasing works: http://www.cs.jhu.edu/~xuchen/paper/Yao2010Master.pdf
Linguistics
Grammars
Montague Grammar
- The Proper Treatment of Quanitifiers, Montague 1973
- One useful interpretation of it
- Good description of the challenges of Montague Grammar:
- Lambda operators in Prolog: https://www.complang.tuwien.ac.at/ulrich/Prolog-inedit/ISO-Hiord
- http://people.umass.edu/partee/docs/MontagueGrammarElsevier.PDF
- Another: http://www-personal.umich.edu/~akao/NLP_Paper.htm
- Another: https://plato.stanford.edu/entries/montague-semantics/
(Neo) Davidsonian Grammar
- Davidson, Donald. 1967. The logical form of action sentences
- Good background on neodavidson: http://verbs.colorado.edu/~mpalmer/Ling7800/ParsonsMP.pdf
- Very good summary that motivates why you need the event argument: https://user.phil-fak.uni-duesseldorf.de/~filip/Davidson.67.pdf
- Another great summary: https://people.umass.edu/partee/NZ_2006/NZ20TenseAspectEvents.pdf
Events in Linguistics
- Very interesting background: http://www.linguist.org/files/History-of-Events.pdf
- REALLY useful: http://www.sfu.ca/~dearmond/322/LF-Sentence.pdf
- This seems to explain where events are used for adjectives: https://core.ac.uk/download/pdf/14509892.pdf
- OK, this really gets to the root of why adjectives have them: http://semantics.uchicago.edu/kennedy/docs/routledge.pdf
Generalized Quantifiers
- Basic description: http://ling-blogs.bu.edu/lx502s13/files/2013/04/LX502_RestrictedQuantifiers.pdf
- Good Overview
- Another Good Overview
- Another: https://sites.google.com/site/heathersusanburnett/home/teaching/egg2012-quantification
- This (surprisingly) has a great description of Quantifiers in logic and NL: https://www.encyclopedia.com/humanities/encyclopedias-almanacs-transcripts-and-maps/quantifiers-natural-language
- This describes how restriction and scope works: http://ling-blogs.bu.edu/lx502s13/files/2013/04/LX502_RestrictedQuantifiers.pdf
- The end of this describes a lot of good stuff! http://sharif.edu/~sani/courses/nlp/lec6.pdf
- Converting “every” “most” etc to logic. “most” is not first order! https://people.umass.edu/partee/409/Logic_and_Natural_Language.pdf
Morphosemantic Properties
- Morphosemantic Properties:
- Tense - location in time
- Aspect - fabric of time: a single block of time, continuous flow of time, or repetitive occurrence
- Mood - degree of necessity, obligation, probability, ability
- From: https://en.wikipedia.org/wiki/Grammatical_mood
- Some examples of moods are indicative, interrogative, imperative, subjunctive, injunctive, optative, and potential.
Using Logic With Natural Language
Composability
Natural Language Generation
- Natural Language Generation
- SimpleNLG: This is what I’ve been using and it really is nice to use and simple (I’ve actually been using the Python port of it). It also has some support for languages besides English.
- KPML: The English grammar it uses has been going since the 80s! Much richer, also multilingual. A much bigger commitment to understand and use.
Natural Language Understanding/Processing
Good example of why we need deep (i.e. full understanding) vs. shallow: https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1162/handouts/Computational-Semantics.pdf
- Also a good summary of existing approaches
NLP Background:
- https://www.tutorialspoint.com/natural_language_processing/index.htm
- https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3168328/
- https://www.kdnuggets.com/2019/02/deep-learning-nlp-rnn-cnn.html
- https://www.analyticsvidhya.com/blog/2019/03/pretrained-models-get-started-nlp/
- https://towardsdatascience.com/natural-language-processing-from-basics-to-using-rnn-and-lstm-ef6779e4ae66
Logic
- Predicate Logic Overview
- Situational Calculus: representation and reasoning about time, fluents, and the use of logic for representing knowledge in a computer.[12][13]
- Non-monatonic logic (https://en.wikipedia.org/wiki/Non-monotonic_logic): system associates facts asserted with the rules and facts used to justify them and as those facts change updates the dependent knowledge as well.
Conceptual Graphs
- Initial Definition: http://www.jfsowa.com/pubs/cg1976.pdf
- Intro: http://homepages.gold.ac.uk/polovina/publications/46040001-SP-Intro-to-CG.pdf
- Deeper intro: http://www.jfsowa.com/cg/cg_hbook.pdf
- http://www.cs.fsu.edu/~lacher/courses/CAP5605/lectures/SixthEdChap7.ppt
- More info: https://www.deepdyve.com/lp/oxford-university-press/conceptual-graphs-and-first-order-logic-zBiNtoJHSJ
- Representing Conceptual graphs in prolog: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.48.6204&rep=rep1&type=pdf
Ontologies
- How to build an ontology: https://protege.stanford.edu/publications/ontology_development/ontology101-noy-mcguinness.html
- OWL
- Members of the OWL family have model theoretic formal semantics, and so have strong logical foundations.
- Has wide support including Prolog
- Critiques of OWL
- OPM seems to be a good basic model: https://en.wikipedia.org/wiki/Object_Process_Methodology
Naïve Physics (what most humans have as a model of physics)
- Naïve Physics Manifesto: http://ontology.buffalo.edu/smith/articles/naivephysics.html
- Naïve Physics Perplex: https://pdfs.semanticscholar.org/0cb8/9b2135496e5ab4d8fef97866a6d9c88b1ba4.pdf
- Egg Cracking: http://www-formal.stanford.edu/leora/commonsense/eggcracking.html
- Formalization of motion: http://www.qrg.northwestern.edu/papers/Files/QRG_Dist_Files/QRG_1980/2_122_Forbus_1980_Spatial_Qualitative_Aspects.pdf
Common Sense Reasoning/Scaled Data
- Keywords: Knowledge representation and reasoning (KR², KR&R)
- Cyc (referenced often, hard to find real data)
- Parsing limited english into logic statements so we can train a system: https://github.com/tiantiangao7/kalm/blob/master/docs/KALM_ODBASE18.pdf
- Uses Framenet and BebelNet to get stuff done
- https://en.wikipedia.org/wiki/Semantic_reasoner
- FrameNet: https://framenet.icsi.berkeley.edu/fndrupal/framenet_search
- WordNet: WordNet is the most commonly used computational lexicon of English for word sense disambiguation (WSD),
- A common use of WordNet is to determine the similarity between words.
- Just using synnets for finding synonyms is not enough
- https://www.nltk.org/howto/wordnet.html
- Conceptnet
- Using ConceptNet to Teach Common Sense to an Automated Theorem Prover: http://arcade2019.net/pdfs/6.pdf
- BabelNet: Take wordnet and apply to wikipedia
- Gellish English dictionary
- The Gellish English Dictionary-Taxonomy is an example of an open-source “smart” electronic dictionary, in which concepts are arranged in a subtype-supertype hierarchy, thus forming a taxonomy.
- Putting Pieces Together: Combining FrameNet, VerbNet and WordNet for Robust Semantic Parsing
Converting English into Logic
- Chat 80: https://era.ed.ac.uk/bitstream/handle/1842/6646/Pereira1982.pdf
- See how well/natural Attempto is in turning english into Prolog logic:
- Natural Semantic metalanguage: https://intranet.secure.griffith.edu.au/schools-departments/natural-semantic-metalanguage
- Deep Semantic Interpretations Of Legal Texts using DCG and FOL logic: https://www.cs.rutgers.edu/~mccarty/research/icail07-acm.pdf
- Prolog and NLU: https://www.researchgate.net/publication/2827995_Natural-Language_Interpretation_in_Prolog
- Prolog and NLU: http://www.mtome.com/Publications/PNLA/prolog-digital.pdf
- https://www.researchgate.net/publication/2262147_Masquesql–_An_Efficient_and_Portable_Natural_Language_Query_Interface_for_Relational_Databases
- What to do and how to do it: Translating Natural Language Directives into Temporal and Dynamic Logic: https://hrilab.tufts.edu/publications/dzifcaketal09icra.pdf
- Commanding a robot: https://homes.cs.washington.edu/~lsz/papers/mhzf-iser12.pdf
Micro Worlds
- Zoo World: https://www.ida.liu.se/ext/caisor/pm-archive/krf/021/PM-krf-021.pdf
- Logic Modeling Workshop: https://www.ida.liu.se/ext/etai/lmw/
- Processing language defining the world: https://www.ijcai.org/Proceedings/81-1/Papers/001.pdf
- SOME BASIC MECHANISMS FOR COMMON SENSE REASONING ABOUT STORIES ENVIRONMENTS
Creating Plots/Narratives
- Basic model of plots, how to generate a good story: http://www.erasmatazz.com/library/the-journal-of-computer/jcgd-volume-7/flawed-methods-for-interact.html
- Survey from 2013: https://nil.cs.uno.edu/publications/papers/roberts2008survey.pdf
- Great pointers to lots of data:
- Erasmatron: http://ifwiki.org/index.php/Erasmatron
- Scheherazade IF (Boyang Li and Mark O. Riedl): https://www.cc.gatech.edu/~riedl/pubs/aaai15.pdf
- Build simple narratives from plot graphs:
- Generates partially ordered plots using mechanical turk
- Dilemma Based narrative: http://www.aaai.org/Papers/AIIDE/2007/AIIDE07-001.pdf
- Good for generating infinite stories, limited vocab and storytelling text
- User is coerced into doing things to make a dilemma happen
- On being passed a dilemma the planner finds all plans to achieve this dilemma given the current storyworld state and background knowledge.
- Toward Automated Quest Generation in Text-Adventure Games: https://deepai.org/publication/toward-automated-quest-generation-in-text-adventure-games
- Improvisational Story Elements: https://nips2017creativity.github.io/doc/Improvisational_Agents.pdf
- Tale-spin: https://www.cs.utah.edu/nlp/papers/talespin-ijcai77.pdf
- Data-Driven Personalized Drama Management (Hong Yu and Mark O. Riedl): https://www.cc.gatech.edu/~riedl/pubs/aiide13.pdf
- Façade drama manager: https://pdfs.semanticscholar.org/a5d2/af2a518e2c74761bdc3d976657ac48c9d2f8.pdf
- UNIVERSE: https://nil.cs.uno.edu/publications/papers/lebowitz1985story.pdf
- Comme il faut: https://nil.cs.uno.edu/publications/papers/mccoy2014cif.pdf
- Emotional behavior that is generated
Dialog
- ChatScript: https://sourceforge.net/projects/chatscript/
- The Second Conversational Intelligence Challenge (ConvAI2): https://arxiv.org/pdf/1902.00098.pdf
- Learning to Speak and Act in a Fantasy Text Adventure Game: https://arxiv.org/pdf/1903.03094.pdf
- https://parl.ai/projects/light/
- Personalizing Dialogue Agents: I have a dog, do you have pets too?: https://arxiv.org/pdf/1801.07243.pdf
- POMDP-Based Statistical Spoken Dialog Systems: A Review: http://cs.brown.edu/courses/csci2951-k/papers/young13.pdf
- Bruce Wilcox Chatbot:https://en.wikipedia.org/wiki/Bruce_Wilcox
- Breakdown of games with dialog: https://emshort.blog/2018/09/11/mailbag-deep-conversation/
- Conversation modeling: https://emshort.blog/2019/04/16/mailbag/#more-39174
- Character Engine: https://www.spiritai.com/character-engine/
- Scripts: A script is a structured representation describing a stereotyped sequence of events in a particular context. The script was originally designed by Schank and his research group 1Schank and Abelson 1977)
Interactive Fiction
- https://emshort.blog/
- Discussion of depth of parsing in IF from Emily Short: https://emshort.blog/2010/06/07/so-do-we-need-this-parser-thing-anyway/
- List of IF authoring systems: http://www.ifwiki.org/index.php/Category:Authoring_system
The Inform 7 Platform
- Introduction
- Background on Implementation
Games and Systems
- Chat 80
- http://aidungeon.io
- “Mastering the Dungeon” a mechanical turk AI text adventure
- https://arxiv.org/pdf/1711.07950v2.pdf
- “The world is represented as a set of objects, along with directed typed edges indicating the relations between them. The set of possible actions of an agent are then defined as updates to the graph structure. “
- Bot Colony: https://store.steampowered.com/app/263040/Bot_Colony/
- https://talktotransformer.com
- https://tccoxon.itch.io/dreamer
- Blockchain adventure where you pay to add to the story