Import LLVM 8.0.0 release including clang, lld and lldb.

2 files changed, 303 insertions, 440 deletions

diff --git a/gnu/llvm/docs/tutorial/BuildingAJIT1.rst b/gnu/llvm/docs/tutorial/BuildingAJIT1.rst
index 2b83df42fc2..fcb755bd286 100644
--- a/gnu/llvm/docs/tutorial/BuildingAJIT1.rst
+++ b/gnu/llvm/docs/tutorial/BuildingAJIT1.rst
@@ -8,18 +8,19 @@ Building a JIT: Starting out with KaleidoscopeJIT
 Chapter 1 Introduction
 ======================
 
-**Warning: This text is currently out of date due to ORC API updates.**
+**Warning: This tutorial is currently being updated to account for ORC API
+changes. Only Chapters 1 and 2 are up-to-date.**
 
-**The example code has been updated and can be used. The text will be updated
-once the API churn dies down.**
+**Example code from Chapters 3 to 5 will compile and run, but has not been
+updated**
 
 Welcome to Chapter 1 of the "Building an ORC-based JIT in LLVM" tutorial. This
 tutorial runs through the implementation of a JIT compiler using LLVM's
 On-Request-Compilation (ORC) APIs. It begins with a simplified version of the
 KaleidoscopeJIT class used in the
 `Implementing a language with LLVM <LangImpl01.html>`_ tutorials and then
-introduces new features like optimization, lazy compilation and remote
-execution.
+introduces new features like concurrent compilation, optimization, lazy
+compilation and remote execution.
 
 The goal of this tutorial is to introduce you to LLVM's ORC JIT APIs, show how
 these APIs interact with other parts of LLVM, and to teach you how to recombine
@@ -45,11 +46,9 @@ The structure of the tutorial is:
 - `Chapter #5 <BuildingAJIT5.html>`_: Add process isolation by JITing code into
   a remote process with reduced privileges using the JIT Remote APIs.
 
-To provide input for our JIT we will use the Kaleidoscope REPL from
-`Chapter 7 <LangImpl07.html>`_ of the "Implementing a language in LLVM tutorial",
-with one minor modification: We will remove the FunctionPassManager from the
-code for that chapter and replace it with optimization support in our JIT class
-in Chapter #2.
+To provide input for our JIT we will use a lightly modified version of the
+Kaleidoscope REPL from `Chapter 7 <LangImpl07.html>`_ of the "Implementing a
+language in LLVM tutorial".
 
 Finally, a word on API generations: ORC is the 3rd generation of LLVM JIT API.
 It was preceded by MCJIT, and before that by the (now deleted) legacy JIT.
@@ -63,32 +62,29 @@ JIT API Basics
 
 The purpose of a JIT compiler is to compile code "on-the-fly" as it is needed,
 rather than compiling whole programs to disk ahead of time as a traditional
-compiler does. To support that aim our initial, bare-bones JIT API will be:
+compiler does. To support that aim our initial, bare-bones JIT API will have
+just two functions:
 
-1. Handle addModule(Module &M) -- Make the given IR module available for
-   execution.
-2. JITSymbol findSymbol(const std::string &Name) -- Search for pointers to
+1. ``Error addModule(std::unique_ptr<Module> M)``: Make the given IR module
+   available for execution.
+2. ``Expected<JITEvaluatedSymbol> lookup()``: Search for pointers to
    symbols (functions or variables) that have been added to the JIT.
-3. void removeModule(Handle H) -- Remove a module from the JIT, releasing any
-   memory that had been used for the compiled code.
 
 A basic use-case for this API, executing the 'main' function from a module,
 will look like:
 
 .. code-block:: c++
 
-  std::unique_ptr<Module> M = buildModule();
   JIT J;
-  Handle H = J.addModule(*M);
-  int (*Main)(int, char*[]) = (int(*)(int, char*[]))J.getSymbolAddress("main");
+  J.addModule(buildModule());
+  auto *Main = (int(*)(int, char*[]))J.lookup("main").getAddress();
   int Result = Main();
-  J.removeModule(H);
 
 The APIs that we build in these tutorials will all be variations on this simple
-theme. Behind the API we will refine the implementation of the JIT to add
-support for optimization and lazy compilation. Eventually we will extend the
-API itself to allow higher-level program representations (e.g. ASTs) to be
-added to the JIT.
+theme. Behind this API we will refine the implementation of the JIT to add
+support for concurrent compilation, optimization and lazy compilation.
+Eventually we will extend the API itself to allow higher-level program
+representations (e.g. ASTs) to be added to the JIT.
 
 KaleidoscopeJIT
 ===============
@@ -100,12 +96,10 @@ the REPL code from `Chapter 7 <LangImpl07.html>`_ of that tutorial to supply the
 input for our JIT: Each time the user enters an expression the REPL will add a
 new IR module containing the code for that expression to the JIT. If the
 expression is a top-level expression like '1+1' or 'sin(x)', the REPL will also
-use the findSymbol method of our JIT class find and execute the code for the
-expression, and then use the removeModule method to remove the code again
-(since there's no way to re-invoke an anonymous expression). In later chapters
-of this tutorial we'll modify the REPL to enable new interactions with our JIT
-class, but for now we will take this setup for granted and focus our attention on
-the implementation of our JIT itself.
+use the lookup method of our JIT class find and execute the code for the
+expression. In later chapters of this tutorial we will modify the REPL to enable
+new interactions with our JIT class, but for now we will take this setup for
+granted and focus our attention on the implementation of our JIT itself.
 
 Our KaleidoscopeJIT class is defined in the KaleidoscopeJIT.h header. After the
 usual include guards and #includes [2]_, we get to the definition of our class:
@@ -115,216 +109,155 @@ usual include guards and #includes [2]_, we get to the definition of our class:
   #ifndef LLVM_EXECUTIONENGINE_ORC_KALEIDOSCOPEJIT_H
   #define LLVM_EXECUTIONENGINE_ORC_KALEIDOSCOPEJIT_H
 
-  #include "llvm/ADT/STLExtras.h"
-  #include "llvm/ExecutionEngine/ExecutionEngine.h"
+  #include "llvm/ADT/StringRef.h"
   #include "llvm/ExecutionEngine/JITSymbol.h"
-  #include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
-  #include "llvm/ExecutionEngine/SectionMemoryManager.h"
   #include "llvm/ExecutionEngine/Orc/CompileUtils.h"
+  #include "llvm/ExecutionEngine/Orc/Core.h"
+  #include "llvm/ExecutionEngine/Orc/ExecutionUtils.h"
   #include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
-  #include "llvm/ExecutionEngine/Orc/LambdaResolver.h"
+  #include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
   #include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h"
+  #include "llvm/ExecutionEngine/SectionMemoryManager.h"
   #include "llvm/IR/DataLayout.h"
-  #include "llvm/IR/Mangler.h"
-  #include "llvm/Support/DynamicLibrary.h"
-  #include "llvm/Support/raw_ostream.h"
-  #include "llvm/Target/TargetMachine.h"
-  #include <algorithm>
+  #include "llvm/IR/LLVMContext.h"
   #include <memory>
-  #include <string>
-  #include <vector>
 
   namespace llvm {
   namespace orc {
 
   class KaleidoscopeJIT {
   private:
-    std::unique_ptr<TargetMachine> TM;
-    const DataLayout DL;
+    ExecutionSession ES;
     RTDyldObjectLinkingLayer ObjectLayer;
-    IRCompileLayer<decltype(ObjectLayer), SimpleCompiler> CompileLayer;
+    IRCompileLayer CompileLayer;
+
+    DataLayout DL;
+    MangleAndInterner Mangle;
+    ThreadSafeContext Ctx;
 
   public:
-    using ModuleHandle = decltype(CompileLayer)::ModuleHandleT;
-
-Our class begins with four members: A TargetMachine, TM, which will be used to
-build our LLVM compiler instance; A DataLayout, DL, which will be used for
-symbol mangling (more on that later), and two ORC *layers*: an
-RTDyldObjectLinkingLayer and a CompileLayer. We'll be talking more about layers
-in the next chapter, but for now you can think of them as analogous to LLVM
-Passes: they wrap up useful JIT utilities behind an easy to compose interface.
-The first layer, ObjectLayer, is the foundation of our JIT: it takes in-memory
-object files produced by a compiler and links them on the fly to make them
-executable. This JIT-on-top-of-a-linker design was introduced in MCJIT, however
-the linker was hidden inside the MCJIT class. In ORC we expose the linker so
-that clients can access and configure it directly if they need to. In this
-tutorial our ObjectLayer will just be used to support the next layer in our
-stack: the CompileLayer, which will be responsible for taking LLVM IR, compiling
-it, and passing the resulting in-memory object files down to the object linking
-layer below.
-
-That's it for member variables, after that we have a single typedef:
-ModuleHandle. This is the handle type that will be returned from our JIT's
-addModule method, and can be passed to the removeModule method to remove a
-module. The IRCompileLayer class already provides a convenient handle type
-(IRCompileLayer::ModuleHandleT), so we just alias our ModuleHandle to this.
+    KaleidoscopeJIT(JITTargetMachineBuilder JTMB, DataLayout DL)
+        : ObjectLayer(ES,
+                      []() { return llvm::make_unique<SectionMemoryManager>(); }),
+          CompileLayer(ES, ObjectLayer, ConcurrentIRCompiler(std::move(JTMB))),
+          DL(std::move(DL)), Mangle(ES, this->DL),
+          Ctx(llvm::make_unique<LLVMContext>()) {
+      ES.getMainJITDylib().setGenerator(
+          cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL)));
+    }
+
+Our class begins with six member variables: An ExecutionSession member, ``ES``,
+which provides context for our running JIT'd code (including the string pool,
+global mutex, and error reporting facilities); An RTDyldObjectLinkingLayer,
+``ObjectLayer``, that can be used to add object files to our JIT (though we will
+not use it directly); An IRCompileLayer, ``CompileLayer``, that can be used to
+add LLVM Modules to our JIT (and which builds on the ObjectLayer), A DataLayout
+and MangleAndInterner, ``DL`` and ``Mangle``, that will be used for symbol mangling
+(more on that later); and finally an LLVMContext that clients will use when
+building IR files for the JIT.
+
+Next up we have our class constructor, which takes a `JITTargetMachineBuilder``
+that will be used by our IRCompiler, and a ``DataLayout`` that we will use to
+initialize our DL member. The constructor begins by initializing our
+ObjectLayer.  The ObjectLayer requires a reference to the ExecutionSession, and
+a function object that will build a JIT memory manager for each module that is
+added (a JIT memory manager manages memory allocations, memory permissions, and
+registration of exception handlers for JIT'd code). For this we use a lambda
+that returns a SectionMemoryManager, an off-the-shelf utility that provides all
+the basic memory management functionality required for this chapter. Next we
+initialize our CompileLayer. The CompileLayer needs three things: (1) A
+reference to the ExecutionSession, (2) A reference to our object layer, and (3)
+a compiler instance to use to perform the actual compilation from IR to object
+files. We use the off-the-shelf ConcurrentIRCompiler utility as our compiler,
+which we construct using this constructor's JITTargetMachineBuilder argument.
+The ConcurrentIRCompiler utility will use the JITTargetMachineBuilder to build
+llvm TargetMachines (which are not thread safe) as needed for compiles. After
+this, we initialize our supporting members: ``DL``, ``Mangler`` and ``Ctx`` with
+the input DataLayout, the ExecutionSession and DL member, and a new default
+constucted LLVMContext respectively. Now that our members have been initialized,
+so the one thing that remains to do is to tweak the configuration of the
+*JITDylib* that we will store our code in. We want to modify this dylib to
+contain not only the symbols that we add to it, but also the symbols from our
+REPL process as well. We do this by attaching a
+``DynamicLibrarySearchGenerator`` instance using the
+``DynamicLibrarySearchGenerator::GetForCurrentProcess`` method.
+
 
 .. code-block:: c++
 
-  KaleidoscopeJIT()
-      : TM(EngineBuilder().selectTarget()), DL(TM->createDataLayout()),
-        ObjectLayer([]() { return std::make_shared<SectionMemoryManager>(); }),
-        CompileLayer(ObjectLayer, SimpleCompiler(*TM)) {
-    llvm::sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
-  }
+  static Expected<std::unique_ptr<KaleidoscopeJIT>> Create() {
+    auto JTMB = JITTargetMachineBuilder::detectHost();
 
-  TargetMachine &getTargetMachine() { return *TM; }
-
-Next up we have our class constructor. We begin by initializing TM using the
-EngineBuilder::selectTarget helper method which constructs a TargetMachine for
-the current process. Then we use our newly created TargetMachine to initialize
-DL, our DataLayout. After that we need to initialize our ObjectLayer. The
-ObjectLayer requires a function object that will build a JIT memory manager for
-each module that is added (a JIT memory manager manages memory allocations,
-memory permissions, and registration of exception handlers for JIT'd code). For
-this we use a lambda that returns a SectionMemoryManager, an off-the-shelf
-utility that provides all the basic memory management functionality required for
-this chapter. Next we initialize our CompileLayer. The CompileLayer needs two
-things: (1) A reference to our object layer, and (2) a compiler instance to use
-to perform the actual compilation from IR to object files. We use the
-off-the-shelf SimpleCompiler instance for now. Finally, in the body of the
-constructor, we call the DynamicLibrary::LoadLibraryPermanently method with a
-nullptr argument. Normally the LoadLibraryPermanently method is called with the
-path of a dynamic library to load, but when passed a null pointer it will 'load'
-the host process itself, making its exported symbols available for execution.
+    if (!JTMB)
+      return JTMB.takeError();
 
-.. code-block:: c++
+    auto DL = JTMB->getDefaultDataLayoutForTarget();
+    if (!DL)
+      return DL.takeError();
 
-  ModuleHandle addModule(std::unique_ptr<Module> M) {
-    // Build our symbol resolver:
-    // Lambda 1: Look back into the JIT itself to find symbols that are part of
-    //           the same "logical dylib".
-    // Lambda 2: Search for external symbols in the host process.
-    auto Resolver = createLambdaResolver(
-        [&](const std::string &Name) {
-          if (auto Sym = CompileLayer.findSymbol(Name, false))
-            return Sym;
-          return JITSymbol(nullptr);
-        },
-        [](const std::string &Name) {
-          if (auto SymAddr =
-                RTDyldMemoryManager::getSymbolAddressInProcess(Name))
-            return JITSymbol(SymAddr, JITSymbolFlags::Exported);
-          return JITSymbol(nullptr);
-        });
-
-    // Add the set to the JIT with the resolver we created above and a newly
-    // created SectionMemoryManager.
-    return cantFail(CompileLayer.addModule(std::move(M),
-                                           std::move(Resolver)));
+    return llvm::make_unique<KaleidoscopeJIT>(std::move(*JTMB), std::move(*DL));
   }
 
-Now we come to the first of our JIT API methods: addModule. This method is
-responsible for adding IR to the JIT and making it available for execution. In
-this initial implementation of our JIT we will make our modules "available for
-execution" by adding them straight to the CompileLayer, which will immediately
-compile them. In later chapters we will teach our JIT to defer compilation
-of individual functions until they're actually called.
-
-To add our module to the CompileLayer we need to supply both the module and a
-symbol resolver. The symbol resolver is responsible for supplying the JIT with
-an address for each *external symbol* in the module we are adding. External
-symbols are any symbol not defined within the module itself, including calls to
-functions outside the JIT and calls to functions defined in other modules that
-have already been added to the JIT. (It may seem as though modules added to the
-JIT should know about one another by default, but since we would still have to
-supply a symbol resolver for references to code outside the JIT it turns out to
-be easier to re-use this one mechanism for all symbol resolution.) This has the
-added benefit that the user has full control over the symbol resolution
-process. Should we search for definitions within the JIT first, then fall back
-on external definitions? Or should we prefer external definitions where
-available and only JIT code if we don't already have an available
-implementation? By using a single symbol resolution scheme we are free to choose
-whatever makes the most sense for any given use case.
-
-Building a symbol resolver is made especially easy by the *createLambdaResolver*
-function. This function takes two lambdas [3]_ and returns a JITSymbolResolver
-instance. The first lambda is used as the implementation of the resolver's
-findSymbolInLogicalDylib method, which searches for symbol definitions that
-should be thought of as being part of the same "logical" dynamic library as this
-Module. If you are familiar with static linking: this means that
-findSymbolInLogicalDylib should expose symbols with common linkage and hidden
-visibility. If all this sounds foreign you can ignore the details and just
-remember that this is the first method that the linker will use to try to find a
-symbol definition. If the findSymbolInLogicalDylib method returns a null result
-then the linker will call the second symbol resolver method, called findSymbol,
-which searches for symbols that should be thought of as external to (but
-visibile from) the module and its logical dylib. In this tutorial we will adopt
-the following simple scheme: All modules added to the JIT will behave as if they
-were linked into a single, ever-growing logical dylib. To implement this our
-first lambda (the one defining findSymbolInLogicalDylib) will just search for
-JIT'd code by calling the CompileLayer's findSymbol method. If we don't find a
-symbol in the JIT itself we'll fall back to our second lambda, which implements
-findSymbol. This will use the RTDyldMemoryManager::getSymbolAddressInProcess
-method to search for the symbol within the program itself. If we can't find a
-symbol definition via either of these paths, the JIT will refuse to accept our
-module, returning a "symbol not found" error.
-
-Now that we've built our symbol resolver, we're ready to add our module to the
-JIT. We do this by calling the CompileLayer's addModule method. The addModule
-method returns an ``Expected<CompileLayer::ModuleHandle>``, since in more
-advanced JIT configurations it could fail. In our basic configuration we know
-that it will always succeed so we use the cantFail utility to assert that no
-error occurred, and extract the handle value. Since we have already typedef'd
-our ModuleHandle type to be the same as the CompileLayer's handle type, we can
-return the unwrapped handle directly.
+  const DataLayout &getDataLayout() const { return DL; }
 
-.. code-block:: c++
+  LLVMContext &getContext() { return *Ctx.getContext(); }
 
-  JITSymbol findSymbol(const std::string Name) {
-    std::string MangledName;
-    raw_string_ostream MangledNameStream(MangledName);
-    Mangler::getNameWithPrefix(MangledNameStream, Name, DL);
-    return CompileLayer.findSymbol(MangledNameStream.str(), true);
-  }
+Next we have a named constructor, ``Create``, which will build a KaleidoscopeJIT
+instance that is configured to generate code for our host process. It does this
+by first generating a JITTargetMachineBuilder instance using that clases's
+detectHost method and then using that instance to generate a datalayout for
+the target process. Each of these operations can fail, so each returns its
+result wrapped in an Expected value [3]_ that we must check for error before
+continuing. If both operations succeed we can unwrap their results (using the
+dereference operator) and pass them into KaleidoscopeJIT's constructor on the
+last line of the function.
+
+Following the named constructor we have the ``getDataLayout()`` and
+``getContext()`` methods. These are used to make data structures created and
+managed by the JIT (especially the LLVMContext) available to the REPL code that
+will build our IR modules.
 
-  JITTargetAddress getSymbolAddress(const std::string Name) {
-    return cantFail(findSymbol(Name).getAddress());
+.. code-block:: c++
+
+  void addModule(std::unique_ptr<Module> M) {
+    cantFail(CompileLayer.add(ES.getMainJITDylib(),
+                              ThreadSafeModule(std::move(M), Ctx)));
   }
 
-  void removeModule(ModuleHandle H) {
-    cantFail(CompileLayer.removeModule(H));
+  Expected<JITEvaluatedSymbol> lookup(StringRef Name) {
+    return ES.lookup({&ES.getMainJITDylib()}, Mangle(Name.str()));
   }
 
-Now that we can add code to our JIT, we need a way to find the symbols we've
-added to it. To do that we call the findSymbol method on our CompileLayer, but
-with a twist: We have to *mangle* the name of the symbol we're searching for
-first. The ORC JIT components use mangled symbols internally the same way a
-static compiler and linker would, rather than using plain IR symbol names. This
-allows JIT'd code to interoperate easily with precompiled code in the
-application or shared libraries. The kind of mangling will depend on the
-DataLayout, which in turn depends on the target platform. To allow us to remain
-portable and search based on the un-mangled name, we just re-produce this
-mangling ourselves.
-
-Next we have a convenience function, getSymbolAddress, which returns the address
-of a given symbol. Like CompileLayer's addModule function, JITSymbol's getAddress
-function is allowed to fail [4]_, however we know that it will not in our simple
-example, so we wrap it in a call to cantFail.
-
-We now come to the last method in our JIT API: removeModule. This method is
-responsible for destructing the MemoryManager and SymbolResolver that were
-added with a given module, freeing any resources they were using in the
-process. In our Kaleidoscope demo we rely on this method to remove the module
-representing the most recent top-level expression, preventing it from being
-treated as a duplicate definition when the next top-level expression is
-entered. It is generally good to free any module that you know you won't need
-to call further, just to free up the resources dedicated to it. However, you
-don't strictly need to do this: All resources will be cleaned up when your
-JIT class is destructed, if they haven't been freed before then. Like
-``CompileLayer::addModule`` and ``JITSymbol::getAddress``, removeModule may
-fail in general but will never fail in our example, so we wrap it in a call to
-cantFail.
+Now we come to the first of our JIT API methods: addModule. This method is
+responsible for adding IR to the JIT and making it available for execution. In
+this initial implementation of our JIT we will make our modules "available for
+execution" by adding them to the CompileLayer, which will it turn store the
+Module in the main JITDylib. This process will create new symbol table entries
+in the JITDylib for each definition in the module, and will defer compilation of
+the module until any of its definitions is looked up. Note that this is not lazy
+compilation: just referencing a definition, even if it is never used, will be
+enough to trigger compilation. In later chapters we will teach our JIT to defer
+compilation of functions until they're actually called.  To add our Module we
+must first wrap it in a ThreadSafeModule instance, which manages the lifetime of
+the Module's LLVMContext (our Ctx member) in a thread-friendly way. In our
+example, all modules will share the Ctx member, which will exist for the
+duration of the JIT. Once we switch to concurrent compilation in later chapters
+we will use a new context per module.
+
+Our last method is ``lookup``, which allows us to look up addresses for
+function and variable definitions added to the JIT based on their symbol names.
+As noted above, lookup will implicitly trigger compilation for any symbol
+that has not already been compiled. Our lookup method calls through to
+`ExecutionSession::lookup`, passing in a list of dylibs to search (in our case
+just the main dylib), and the symbol name to search for, with a twist: We have
+to *mangle* the name of the symbol we're searching for first. The ORC JIT
+components use mangled symbols internally the same way a static compiler and
+linker would, rather than using plain IR symbol names. This allows JIT'd code
+to interoperate easily with precompiled code in the application or shared
+libraries. The kind of mangling will depend on the DataLayout, which in turn
+depends on the target platform. To allow us to remain portable and search based
+on the un-mangled name, we just re-produce this mangling ourselves using our
+``Mangle`` member function object.
 
 This brings us to the end of Chapter 1 of Building a JIT. You now have a basic
 but fully functioning JIT stack that you can use to take LLVM IR and make it
@@ -362,42 +295,29 @@ Here is the code:
 .. [2] +-----------------------------+-----------------------------------------------+
        |         File                |               Reason for inclusion            |
        +=============================+===============================================+
-       |      STLExtras.h            | LLVM utilities that are useful when working   |
-       |                             | with the STL.                                 |
+       |        JITSymbol.h          | Defines the lookup result type                |
+       |                             | JITEvaluatedSymbol                            |
        +-----------------------------+-----------------------------------------------+
-       |   ExecutionEngine.h         | Access to the EngineBuilder::selectTarget     |
-       |                             | method.                                       |
+       |       CompileUtils.h        | Provides the SimpleCompiler class.            |
        +-----------------------------+-----------------------------------------------+
-       |                             | Access to the                                 |
-       | RTDyldMemoryManager.h       | RTDyldMemoryManager::getSymbolAddressInProcess|
-       |                             | method.                                       |
+       |           Core.h            | Core utilities such as ExecutionSession and   |
+       |                             | JITDylib.                                     |
        +-----------------------------+-----------------------------------------------+
-       |    CompileUtils.h           | Provides the SimpleCompiler class.            |
+       |      ExecutionUtils.h       | Provides the DynamicLibrarySearchGenerator    |
+       |                             | class.                                        |
        +-----------------------------+-----------------------------------------------+
-       |   IRCompileLayer.h          | Provides the IRCompileLayer class.            |
+       |      IRCompileLayer.h       | Provides the IRCompileLayer class.            |
        +-----------------------------+-----------------------------------------------+
-       |                             | Access the createLambdaResolver function,     |
-       |   LambdaResolver.h          | which provides easy construction of symbol    |
-       |                             | resolvers.                                    |
+       |  JITTargetMachineBuilder.h  | Provides the JITTargetMachineBuilder class.   |
        +-----------------------------+-----------------------------------------------+
-       |  RTDyldObjectLinkingLayer.h | Provides the RTDyldObjectLinkingLayer class.  |
+       | RTDyldObjectLinkingLayer.h  | Provides the RTDyldObjectLinkingLayer class.  |
        +-----------------------------+-----------------------------------------------+
-       |       Mangler.h             | Provides the Mangler class for platform       |
-       |                             | specific name-mangling.                       |
+       |   SectionMemoryManager.h    | Provides the SectionMemoryManager class.      |
        +-----------------------------+-----------------------------------------------+
-       |   DynamicLibrary.h          | Provides the DynamicLibrary class, which      |
-       |                             | makes symbols in the host process searchable. |
+       |        DataLayout.h         | Provides the DataLayout class.                |
        +-----------------------------+-----------------------------------------------+
-       |                             | A fast output stream class. We use the        |
-       |     raw_ostream.h           | raw_string_ostream subclass for symbol        |
-       |                             | mangling                                      |
+       |        LLVMContext.h        | Provides the LLVMContext class.               |
        +-----------------------------+-----------------------------------------------+
-       |   TargetMachine.h           | LLVM target machine description class.        |
-       +-----------------------------+-----------------------------------------------+
-
-.. [3] Actually they don't have to be lambdas, any object with a call operator
-       will do, including plain old functions or std::functions.
 
-.. [4] ``JITSymbol::getAddress`` will force the JIT to compile the definition of
-       the symbol if it hasn't already been compiled, and since the compilation
-       process could fail getAddress must be able to return this failure.
+.. [3] See the ErrorHandling section in the LLVM Programmer's Manual
+       (http://llvm.org/docs/ProgrammersManual.html#error-handling)
\ No newline at end of file
diff --git a/gnu/llvm/docs/tutorial/BuildingAJIT2.rst b/gnu/llvm/docs/tutorial/BuildingAJIT2.rst
index 15c9c3586bc..7d25ccaba0a 100644
--- a/gnu/llvm/docs/tutorial/BuildingAJIT2.rst
+++ b/gnu/llvm/docs/tutorial/BuildingAJIT2.rst
@@ -12,10 +12,11 @@ we welcome any feedback.
 Chapter 2 Introduction
 ======================
 
-**Warning: This text is currently out of date due to ORC API updates.**
+**Warning: This tutorial is currently being updated to account for ORC API
+changes. Only Chapters 1 and 2 are up-to-date.**
 
-**The example code has been updated and can be used. The text will be updated
-once the API churn dies down.**
+**Example code from Chapters 3 to 5 will compile and run, but has not been
+updated**
 
 Welcome to Chapter 2 of the "Building an ORC-based JIT in LLVM" tutorial. In
 `Chapter 1 <BuildingAJIT1.html>`_ of this series we examined a basic JIT
@@ -42,67 +43,49 @@ added to it. In this Chapter we will make optimization a phase of our JIT
 instead. For now this will provide us a motivation to learn more about ORC
 layers, but in the long term making optimization part of our JIT will yield an
 important benefit: When we begin lazily compiling code (i.e. deferring
-compilation of each function until the first time it's run), having
+compilation of each function until the first time it's run) having
 optimization managed by our JIT will allow us to optimize lazily too, rather
 than having to do all our optimization up-front.
 
 To add optimization support to our JIT we will take the KaleidoscopeJIT from
 Chapter 1 and compose an ORC *IRTransformLayer* on top. We will look at how the
 IRTransformLayer works in more detail below, but the interface is simple: the
-constructor for this layer takes a reference to the layer below (as all layers
-do) plus an *IR optimization function* that it will apply to each Module that
-is added via addModule:
+constructor for this layer takes a reference to the execution session and the
+layer below (as all layers do) plus an *IR optimization function* that it will
+apply to each Module that is added via addModule:
 
 .. code-block:: c++
 
   class KaleidoscopeJIT {
   private:
-    std::unique_ptr<TargetMachine> TM;
-    const DataLayout DL;
-    RTDyldObjectLinkingLayer<> ObjectLayer;
-    IRCompileLayer<decltype(ObjectLayer)> CompileLayer;
+    ExecutionSession ES;
+    RTDyldObjectLinkingLayer ObjectLayer;
+    IRCompileLayer CompileLayer;
+    IRTransformLayer TransformLayer;
 
-    using OptimizeFunction =
-        std::function<std::shared_ptr<Module>(std::shared_ptr<Module>)>;
-
-    IRTransformLayer<decltype(CompileLayer), OptimizeFunction> OptimizeLayer;
+    DataLayout DL;
+    MangleAndInterner Mangle;
+    ThreadSafeContext Ctx;
 
   public:
-    using ModuleHandle = decltype(OptimizeLayer)::ModuleHandleT;
-
-    KaleidoscopeJIT()
-        : TM(EngineBuilder().selectTarget()), DL(TM->createDataLayout()),
-          ObjectLayer([]() { return std::make_shared<SectionMemoryManager>(); }),
-          CompileLayer(ObjectLayer, SimpleCompiler(*TM)),
-          OptimizeLayer(CompileLayer,
-                        [this](std::unique_ptr<Module> M) {
-                          return optimizeModule(std::move(M));
-                        }) {
-      llvm::sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
+
+    KaleidoscopeJIT(JITTargetMachineBuilder JTMB, DataLayout DL)
+        : ObjectLayer(ES,
+                      []() { return llvm::make_unique<SectionMemoryManager>(); }),
+          CompileLayer(ES, ObjectLayer, ConcurrentIRCompiler(std::move(JTMB))),
+          TransformLayer(ES, CompileLayer, optimizeModule),
+          DL(std::move(DL)), Mangle(ES, this->DL),
+          Ctx(llvm::make_unique<LLVMContext>()) {
+      ES.getMainJITDylib().setGenerator(
+          cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL)));
     }
 
 Our extended KaleidoscopeJIT class starts out the same as it did in Chapter 1,
-but after the CompileLayer we introduce a typedef for our optimization function.
-In this case we use a std::function (a handy wrapper for "function-like" things)
-from a single unique_ptr<Module> input to a std::unique_ptr<Module> output. With
-our optimization function typedef in place we can declare our OptimizeLayer,
-which sits on top of our CompileLayer.
-
-To initialize our OptimizeLayer we pass it a reference to the CompileLayer
-below (standard practice for layers), and we initialize the OptimizeFunction
-using a lambda that calls out to an "optimizeModule" function that we will
-define below.
-
-.. code-block:: c++
-
-  // ...
-  auto Resolver = createLambdaResolver(
-      [&](const std::string &Name) {
-        if (auto Sym = OptimizeLayer.findSymbol(Name, false))
-          return Sym;
-        return JITSymbol(nullptr);
-      },
-  // ...
+but after the CompileLayer we introduce a new member, TransformLayer, which sits
+on top of our CompileLayer. We initialize our OptimizeLayer with a reference to
+the ExecutionSession and output layer (standard practice for layers), along with
+a *transform function*. For our transform function we supply our classes
+optimizeModule static method.
 
 .. code-block:: c++
 
@@ -111,26 +94,13 @@ define below.
                                           std::move(Resolver)));
   // ...
 
-.. code-block:: c++
-
-  // ...
-  return OptimizeLayer.findSymbol(MangledNameStream.str(), true);
-  // ...
-
-.. code-block:: c++
-
-  // ...
-  cantFail(OptimizeLayer.removeModule(H));
-  // ...
-
-Next we need to replace references to 'CompileLayer' with references to
-OptimizeLayer in our key methods: addModule, findSymbol, and removeModule. In
-addModule we need to be careful to replace both references: the findSymbol call
-inside our resolver, and the call through to addModule.
+Next we need to update our addModule method to replace the call to
+``CompileLayer::add`` with a call to ``OptimizeLayer::add`` instead.
 
 .. code-block:: c++
 
-  std::shared_ptr<Module> optimizeModule(std::shared_ptr<Module> M) {
+  static Expected<ThreadSafeModule>
+  optimizeModule(ThreadSafeModule M, const MaterializationResponsibility &R) {
     // Create a function pass manager.
     auto FPM = llvm::make_unique<legacy::FunctionPassManager>(M.get());
 
@@ -150,12 +120,18 @@ inside our resolver, and the call through to addModule.
   }
 
 At the bottom of our JIT we add a private method to do the actual optimization:
-*optimizeModule*. This function sets up a FunctionPassManager, adds some passes
-to it, runs it over every function in the module, and then returns the mutated
-module. The specific optimizations are the same ones used in
-`Chapter 4 <LangImpl04.html>`_ of the "Implementing a language with LLVM"
-tutorial series. Readers may visit that chapter for a more in-depth
-discussion of these, and of IR optimization in general.
+*optimizeModule*. This function takes the module to be transformed as input (as
+a ThreadSafeModule) along with a reference to a reference to a new class:
+``MaterializationResponsibility``. The MaterializationResponsibility argument
+can be used to query JIT state for the module being transformed, such as the set
+of definitions in the module that JIT'd code is actively trying to call/access.
+For now we will ignore this argument and use a standard optimization
+pipeline. To do this we set up a FunctionPassManager, add some passes to it, run
+it over every function in the module, and then return the mutated module. The
+specific optimizations are the same ones used in `Chapter 4 <LangImpl04.html>`_
+of the "Implementing a language with LLVM" tutorial series. Readers may visit
+that chapter for a more in-depth discussion of these, and of IR optimization in
+general.
 
 And that's it in terms of changes to KaleidoscopeJIT: When a module is added via
 addModule the OptimizeLayer will call our optimizeModule function before passing
@@ -163,148 +139,122 @@ the transformed module on to the CompileLayer below. Of course, we could have
 called optimizeModule directly in our addModule function and not gone to the
 bother of using the IRTransformLayer, but doing so gives us another opportunity
 to see how layers compose. It also provides a neat entry point to the *layer*
-concept itself, because IRTransformLayer turns out to be one of the simplest
-implementations of the layer concept that can be devised:
+concept itself, because IRTransformLayer is one of the simplest layers that
+can be implemented.
 
 .. code-block:: c++
 
-  template <typename BaseLayerT, typename TransformFtor>
-  class IRTransformLayer {
+  // From IRTransformLayer.h:
+  class IRTransformLayer : public IRLayer {
   public:
-    using ModuleHandleT = typename BaseLayerT::ModuleHandleT;
+    using TransformFunction = std::function<Expected<ThreadSafeModule>(
+        ThreadSafeModule, const MaterializationResponsibility &R)>;
 
-    IRTransformLayer(BaseLayerT &BaseLayer,
-                     TransformFtor Transform = TransformFtor())
-      : BaseLayer(BaseLayer), Transform(std::move(Transform)) {}
+    IRTransformLayer(ExecutionSession &ES, IRLayer &BaseLayer,
+                     TransformFunction Transform = identityTransform);
 
-    Expected<ModuleHandleT>
-    addModule(std::shared_ptr<Module> M,
-              std::shared_ptr<JITSymbolResolver> Resolver) {
-      return BaseLayer.addModule(Transform(std::move(M)), std::move(Resolver));
+    void setTransform(TransformFunction Transform) {
+      this->Transform = std::move(Transform);
     }
 
-    void removeModule(ModuleHandleT H) { BaseLayer.removeModule(H); }
-
-    JITSymbol findSymbol(const std::string &Name, bool ExportedSymbolsOnly) {
-      return BaseLayer.findSymbol(Name, ExportedSymbolsOnly);
+    static ThreadSafeModule
+    identityTransform(ThreadSafeModule TSM,
+                      const MaterializationResponsibility &R) {
+      return TSM;
     }
 
-    JITSymbol findSymbolIn(ModuleHandleT H, const std::string &Name,
-                           bool ExportedSymbolsOnly) {
-      return BaseLayer.findSymbolIn(H, Name, ExportedSymbolsOnly);
-    }
+    void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;
 
-    void emitAndFinalize(ModuleHandleT H) {
-      BaseLayer.emitAndFinalize(H);
-    }
+  private:
+    IRLayer &BaseLayer;
+    TransformFunction Transform;
+  };
 
-    TransformFtor& getTransform() { return Transform; }
+  // From IRTransfomrLayer.cpp:
 
-    const TransformFtor& getTransform() const { return Transform; }
+  IRTransformLayer::IRTransformLayer(ExecutionSession &ES,
+                                     IRLayer &BaseLayer,
+                                     TransformFunction Transform)
+      : IRLayer(ES), BaseLayer(BaseLayer), Transform(std::move(Transform)) {}
 
-  private:
-    BaseLayerT &BaseLayer;
-    TransformFtor Transform;
-  };
+  void IRTransformLayer::emit(MaterializationResponsibility R,
+                              ThreadSafeModule TSM) {
+    assert(TSM.getModule() && "Module must not be null");
+
+    if (auto TransformedTSM = Transform(std::move(TSM), R))
+      BaseLayer.emit(std::move(R), std::move(*TransformedTSM));
+    else {
+      R.failMaterialization();
+      getExecutionSession().reportError(TransformedTSM.takeError());
+    }
+  }
 
 This is the whole definition of IRTransformLayer, from
-``llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h``, stripped of its
-comments. It is a template class with two template arguments: ``BaesLayerT`` and
-``TransformFtor`` that provide the type of the base layer and the type of the
-"transform functor" (in our case a std::function) respectively. This class is
-concerned with two very simple jobs: (1) Running every IR Module that is added
-with addModule through the transform functor, and (2) conforming to the ORC
-layer interface. The interface consists of one typedef and five methods:
-
-+------------------+-----------------------------------------------------------+
-|     Interface    |                         Description                       |
-+==================+===========================================================+
-|                  | Provides a handle that can be used to identify a module   |
-| ModuleHandleT    | set when calling findSymbolIn, removeModule, or           |
-|                  | emitAndFinalize.                                          |
-+------------------+-----------------------------------------------------------+
-|                  | Takes a given set of Modules and makes them "available    |
-|                  | for execution". This means that symbols in those modules  |
-|                  | should be searchable via findSymbol and findSymbolIn, and |
-|                  | the address of the symbols should be read/writable (for   |
-|                  | data symbols), or executable (for function symbols) after |
-|                  | JITSymbol::getAddress() is called. Note: This means that  |
-|   addModule      | addModule doesn't have to compile (or do any other        |
-|                  | work) up-front. It *can*, like IRCompileLayer, act        |
-|                  | eagerly, but it can also simply record the module and     |
-|                  | take no further action until somebody calls               |
-|                  | JITSymbol::getAddress(). In IRTransformLayer's case       |
-|                  | addModule eagerly applies the transform functor to        |
-|                  | each module in the set, then passes the resulting set     |
-|                  | of mutated modules down to the layer below.               |
-+------------------+-----------------------------------------------------------+
-|                  | Removes a set of modules from the JIT. Code or data       |
-|  removeModule    | defined in these modules will no longer be available, and |
-|                  | the memory holding the JIT'd definitions will be freed.   |
-+------------------+-----------------------------------------------------------+
-|                  | Searches for the named symbol in all modules that have    |
-|                  | previously been added via addModule (and not yet          |
-|    findSymbol    | removed by a call to removeModule). In                    |
-|                  | IRTransformLayer we just pass the query on to the layer   |
-|                  | below. In our REPL this is our default way to search for  |
-|                  | function definitions.                                     |
-+------------------+-----------------------------------------------------------+
-|                  | Searches for the named symbol in the module set indicated |
-|                  | by the given ModuleHandleT. This is just an optimized     |
-|                  | search, better for lookup-speed when you know exactly     |
-|                  | a symbol definition should be found. In IRTransformLayer  |
-|   findSymbolIn   | we just pass this query on to the layer below. In our     |
-|                  | REPL we use this method to search for functions           |
-|                  | representing top-level expressions, since we know exactly |
-|                  | where we'll find them: in the top-level expression module |
-|                  | we just added.                                            |
-+------------------+-----------------------------------------------------------+
-|                  | Forces all of the actions required to make the code and   |
-|                  | data in a module set (represented by a ModuleHandleT)     |
-|                  | accessible. Behaves as if some symbol in the set had been |
-|                  | searched for and JITSymbol::getSymbolAddress called. This |
-| emitAndFinalize  | is rarely needed, but can be useful when dealing with     |
-|                  | layers that usually behave lazily if the user wants to    |
-|                  | trigger early compilation (for example, to use idle CPU   |
-|                  | time to eagerly compile code in the background).          |
-+------------------+-----------------------------------------------------------+
-
-This interface attempts to capture the natural operations of a JIT (with some
-wrinkles like emitAndFinalize for performance), similar to the basic JIT API
-operations we identified in Chapter 1. Conforming to the layer concept allows
-classes to compose neatly by implementing their behaviors in terms of the these
-same operations, carried out on the layer below. For example, an eager layer
-(like IRTransformLayer) can implement addModule by running each module in the
-set through its transform up-front and immediately passing the result to the
-layer below. A lazy layer, by contrast, could implement addModule by
-squirreling away the modules doing no other up-front work, but applying the
-transform (and calling addModule on the layer below) when the client calls
-findSymbol instead. The JIT'd program behavior will be the same either way, but
-these choices will have different performance characteristics: Doing work
-eagerly means the JIT takes longer up-front, but proceeds smoothly once this is
-done. Deferring work allows the JIT to get up-and-running quickly, but will
-force the JIT to pause and wait whenever some code or data is needed that hasn't
-already been processed.
-
-Our current REPL is eager: Each function definition is optimized and compiled as
-soon as it's typed in. If we were to make the transform layer lazy (but not
-change things otherwise) we could defer optimization until the first time we
-reference a function in a top-level expression (see if you can figure out why,
-then check out the answer below [1]_). In the next chapter, however we'll
-introduce fully lazy compilation, in which function's aren't compiled until
-they're first called at run-time. At this point the trade-offs get much more
+``llvm/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h`` and
+``llvm/lib/ExecutionEngine/Orc/IRTransformLayer.cpp``.  This class is concerned
+with two very simple jobs: (1) Running every IR Module that is emitted via this
+layer through the transform function object, and (2) implementing the ORC
+``IRLayer`` interface (which itself conforms to the general ORC Layer concept,
+more on that below). Most of the class is straightforward: a typedef for the
+transform function, a constructor to initialize the members, a setter for the
+transform function value, and a default no-op transform. The most important
+method is ``emit`` as this is half of our IRLayer interface. The emit method
+applies our transform to each module that it is called on and, if the transform
+succeeds, passes the transformed module to the base layer. If the transform
+fails, our emit function calls
+``MaterializationResponsibility::failMaterialization`` (this JIT clients who
+may be waiting on other threads know that the code they were waiting for has
+failed to compile) and logs the error with the execution session before bailing
+out.
+
+The other half of the IRLayer interface we inherit unmodified from the IRLayer
+class:
+
+.. code-block:: c++
+
+  Error IRLayer::add(JITDylib &JD, ThreadSafeModule TSM, VModuleKey K) {
+    return JD.define(llvm::make_unique<BasicIRLayerMaterializationUnit>(
+        *this, std::move(K), std::move(TSM)));
+  }
+
+This code, from ``llvm/lib/ExecutionEngine/Orc/Layer.cpp``, adds a
+ThreadSafeModule to a given JITDylib by wrapping it up in a
+``MaterializationUnit`` (in this case a ``BasicIRLayerMaterializationUnit``).
+Most layers that derived from IRLayer can rely on this default implementation
+of the ``add`` method.
+
+These two operations, ``add`` and ``emit``, together constitute the layer
+concept: A layer is a way to wrap a portion of a compiler pipeline (in this case
+the "opt" phase of an LLVM compiler) whose API is is opaque to ORC in an
+interface that allows ORC to invoke it when needed. The add method takes an
+module in some input program representation (in this case an LLVM IR module) and
+stores it in the target JITDylib, arranging for it to be passed back to the
+Layer's emit method when any symbol defined by that module is requested. Layers
+can compose neatly by calling the 'emit' method of a base layer to complete
+their work. For example, in this tutorial our IRTransformLayer calls through to
+our IRCompileLayer to compile the transformed IR, and our IRCompileLayer in turn
+calls our ObjectLayer to link the object file produced by our compiler.
+
+
+So far we have learned how to optimize and compile our LLVM IR, but we have not
+focused on when compilation happens. Our current REPL is eager: Each function
+definition is optimized and compiled as soon as it is referenced by any other
+code, regardless of whether it is ever called at runtime. In the next chapter we
+will introduce fully lazy compilation, in which functions are not compiled until
+they are first called at run-time. At this point the trade-offs get much more
 interesting: the lazier we are, the quicker we can start executing the first
-function, but the more often we'll have to pause to compile newly encountered
-functions. If we only code-gen lazily, but optimize eagerly, we'll have a slow
-startup (which everything is optimized) but relatively short pauses as each
-function just passes through code-gen. If we both optimize and code-gen lazily
-we can start executing the first function more quickly, but we'll have longer
-pauses as each function has to be both optimized and code-gen'd when it's first
-executed. Things become even more interesting if we consider interproceedural
-optimizations like inlining, which must be performed eagerly. These are
-complex trade-offs, and there is no one-size-fits all solution to them, but by
-providing composable layers we leave the decisions to the person implementing
-the JIT, and make it easy for them to experiment with different configurations.
+function, but the more often we will have to pause to compile newly encountered
+functions. If we only code-gen lazily, but optimize eagerly, we will have a
+longer startup time (as everything is optimized) but relatively short pauses as
+each function just passes through code-gen. If we both optimize and code-gen
+lazily we can start executing the first function more quickly, but we will have
+longer pauses as each function has to be both optimized and code-gen'd when it
+is first executed. Things become even more interesting if we consider
+interproceedural optimizations like inlining, which must be performed eagerly.
+These are complex trade-offs, and there is no one-size-fits all solution to
+them, but by providing composable layers we leave the decisions to the person
+implementing the JIT, and make it easy for them to experiment with different
+configurations.
 
 `Next: Adding Per-function Lazy Compilation <BuildingAJIT3.html>`_
 
@@ -325,10 +275,3 @@ Here is the code:
 
 .. literalinclude:: ../../examples/Kaleidoscope/BuildingAJIT/Chapter2/KaleidoscopeJIT.h
    :language: c++
-
-.. [1] When we add our top-level expression to the JIT, any calls to functions
-       that we defined earlier will appear to the RTDyldObjectLinkingLayer as
-       external symbols. The RTDyldObjectLinkingLayer will call the SymbolResolver
-       that we defined in addModule, which in turn calls findSymbol on the
-       OptimizeLayer, at which point even a lazy transform layer will have to
-       do its work.