A Live Programming environment is one where objects exist and interact in a continuous manner across updates and manipulations to the program. That is, the program can continue to process inputs and produce outputs even as it is being updated by a human programmer.
Live Programming is heavily used for digital DJs, combining and creating music on the fly with commands, updates, loops, etc. The 'liveness' of the Live Programming environment is also very useful for quick feedback from changes. Live Programming is considerably more 'alive' than a Read Eval Print Loop in that one is actively editing objects or relationships between them as they continue to interact. One is not 'pausing' the program to make an edit. A Live Programming environment doesn't imply a Read Eval Print Loop exists... though one might (if the language provides the needed features) dynamically hook the live program into a console or other GUI to allow the programmer to inject inputs or receive outputs.
Smalltalk Language is one of the original languages providing a Live Programming environment. Lambda Moo would be another. Today, these have branched into Open Croquet and Second Life.
A Programming Language for Live Programming should be designed to localize the 'extent' of syntax errors. That is, an error in one part of the program shouldn't affect the interpretation of other parts of the program. For this reason, Graphical Programming Languages are very useful, as are Object Oriented Programming Languages that allow edits to the source-code of individual methods for individual objects (i.e. prototype-based).
Additionally, to support Live Programming, features such as 'Compile Time Resolution' should generally be avoided, as the semantics of when they are to 'execute' become very unclear, and one may not wish to continuously execute code while editing it. Ensuring that execution of Side Effects is either driven by input or demand-for-output rather than by 'staging' is useful for allowing one to control and manage a Live Programming system.
Also, runtime allocation hurts Live Programming a bit. I.e. if one is creating and deleting objects at runtime, it becomes difficult for a human to track and update the system, and semantics for updates become a bit less clear (i.e. if you change a procedure that creates an object, to which degree should it change the objects it has already created?). Defining a clear Meta Object Protocol can help get past this hurt.
In 2002, I developed (and almost instantly grew disillusioned with) an experimental Live Programming Graphical Programming Language for the Java Platform. See tomatoide.sourceforge.net 
As I wrote at the time, "modifying an application while it executes is an unusual and strangely exhilarating experience."
Interesting. What disillusioned you?
Amongst a number of problems that I believe to be entirely solvable and not worth discussing here, I ran up against two relatively-unsolvable issues that are mentioned particularly in connection with the "icons on strings" style of Graphical Programming Language, but tend to apply to graphical programming in general: First, graphical programming tends to produce a lower density of semantic content per unit of screen area than the corresponding text-based code.[1] The amount of zooming and scrolling needed to grok the code is considerably more than that required for the corresponding text-based code. Second, graphical programming forces the developer to consider the aesthetics and style of code in at least two geometrical dimensions (i.e., positioning of icons and connections in a 2D or 3D space) compared to only one required for text-based code (indentation). It's notable that while icons-on-strings essentially represents code as something like a circuit diagram, complex circuit design tends to favour VHDL, for example, that represents circuits as a text-based programming language.
That said, I'm not dispensing entirely with Graphical Programming Languages, I simply deprecate the applicability of icons-on-strings to general-purpose programming. There are certainly niche areas where GPLs work well (e.g., designing virtual instrumentation, audio processing, or implementing pixel shaders), and it was my interest in applying graphical programming to data-processing and database applications that led me (in part) to develop the Rel Project as a foundation for further work on higher-level (possibly graphical) database languages. It's something to which I will eventually return.
--
[1] This is not to say that the use of icons or pictographs inherently forces lower semantic density. Note that written Chinese, for example, demonstrates higher semantic density than written English. The implications in terms of graphical programming are manifold, and probably do not belong here.
Musical Live Coding:
Quoth : netjam.org
See original on c2.com