As an enhancement to LLVM and a vehicle for both learning LLVM and (possibly)
testing LLVM through an alternate source language, I'm building a front end for
a very simple language named Stacker. It is somewhat Forth based but doesn't
nearly have the expressive power of ANS Forth. The syntax will be familiar to
Forth programmers but the capabilities very limiting.

The initial version will have the following restrictions:
* Input only from stdin
* Output only to stdout
* Only one stack, fixed size
* No floating point support
* Very restricted set of native operations (arithmetic, stackops, I/O).
* No interpretation mode, just compiles.

As such, the language is barely higher level than LLVM Assembly. 

The point is to produce a simple language with a simple grammar and lexical
analyzer that demonstrates how to use LLVM classes to generate bytecode and
execute them with JIT, etc.

This "bug" (enhancement) is provided just to track the work, design, patches,
extensions, etc.

P.S. We might want to create a new Bugzilla "Product" for "Projects" so this bug
can be more appropriately assigned.

Created an attachment (id=31) [details]
Stacker Compiler And Runtime V1.0

This compressed tar file contains the source code, test casts, and sample
programs  for Stacker V1.0. The file contents are ready to be dropped into the
LLVM source tree under the "projects" directory. If I had CVS access, I'd do it
myself :)

Documentation will follow tomorrow (it's 1/2 written).

THE THING TO NOTE FROM THIS EXERCISE:

Note the time stamps on the previous to postings. The first is when I started.
The second is when I was done .. just under four days. 

That ALONE is a testimony to LLVM. I've never gotta a native machine capable
compiler working in so short a time.

Great!! I checked it in.  Here are a couple of things that could be improved:

* In the samples directory, you hardcode in use of the 'Debug' tools.  It
  would be better if you followed the style in the top-level Makefile.rules
  so that building with ENABLE_OPTIMIZED=1 will use the optimized tools.

* I fixed a couple occurances of 'sample' which still existed in the makefiles
  and stuff.  Please be sure that you scrub them to say stacker instead.  :)

* I adjusted the #include in stkrc.cpp, because it didn't work for me.

* I was getting these assertions running stkrc:
stkrc -e -f -o testing.bc testing.st 
Global is external, but doesn't have external linkage!
[1024 x int]* %_stack_
Global is external, but doesn't have external linkage!
long* %_index_
Broken module found, compilation aborted!

... which I fixed with this patch:
http://mail.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20031117/009821.html

* Despite that (or because of it), I still get this when building tests:

gcc -ggdb -L/home/vadve/lattner/cvs/llvm/lib/Debug testing.s -lstkr_runtime -o
eq eq.s
/tmp/ccClusnG.o(.bss+0x0):/home/vadve/lattner/llvm/projects/Stacker/test/eq.s:10:
multiple definition of `_stack_'
/tmp/ccZCK0mV.o(.bss+0x0):/home/vadve/lattner/llvm/projects/Stacker/test/testing.s:10:
first defined here
collect2: ld returned 1 exit status
gmake: *** [eq] Error 1

... This is probably a bug in the X86 code generator, not outputting stuff with
appending linkage correctly or something.  Because if this, I disabled 'test' in
the top-level makefile, so that it does not break our builds.

* The test/Makefile dumps all of the temporary files into the 
  current directory, and doesn't clean them up with the 'make clean' target.
  For other tests, we have taken to compiling into an Output/ directory,
  which makes it easy to separate the generated files from the source of
  the tests.  If you like this style, go ahead and do it, but don't feel
  obligated to.  The samples/Makefile doesn't have a functional clean target.

* Overall I think it would be better to make the stacker tests work more
  like the main LLVM tests: split it into regression and programs, then
  have the programs tests compile in all all of the different modes
  (interpreter, JIT, LLC, CBE).  If you wanted to, I would have no problem
  with having the stacker tests in the main test hierarchy, or just having
  the makefile machinery in the main part of the tree.

Overall, this looks really cool, and should be useful to a lot of people!  I'm
really excited about this.  Thanks Reid!

-Chris

I'm working these problems .. there's bigger issues .. configure.ac isn't right
either.  I'll submit a patch when I get it working out of CVS correctly.

Reid.

Hey!

The Stacker compilation errors you described only occurred *after* I did a cvs
update at the top level. Stacker programs used to compile just fine, before the
update. I'll investigate this further but if you changed the rules on how global
appending linkage works, could you provide some hints as to what is wrong now?

Reid.

Created an attachment (id=32) [details]
Patch To Fix Integration Issues

This patch updates Stacker to resolve the following issues:

1. Link errors that occurred with the latest LLVM code (didn't occur before)
2. Passing command line arguments to a Stacker Program (updated runtime)
3. The "prime" sample now makes use of command line arguments. You can pass
   it numbers and it'll tell you if its a prime.
4. Makefile deficiencies

NOTE: You will need a patched configure.ac and configure script to make this
work correctly. That patch will follow.

Created an attachment (id=33) [details]
Patch To Fix Configuration Of Stacker Directories

This patch fixes problems in autoconf/configure.ac so that it gets the correct
set of directories to build for Stacker.  You'll need to get this patch, run
autoconf on it (in the autoconf subdir), and then reconfigure your object
directory to get Stacker to compiler correctly.

Created an attachment (id=34) [details]
Documentation For Stacker

Here's the first cut at the documentation. After it gets added to CVS, I'll
work on filling in some of the missing blanks (especially the lessons learned
section). 

This file is meant to be dropped into the docs subdirectory. It references
other documents relative to that document. Someone will have to make a link to
it in the other documents. Note that I've used the stylesheets from llvm.css in
the hope that someday that style sheet will be more complete. In any event, XML
1.1 style was used so the formatting should just automagically fix itself when
llvm.css is updated.

I'm going to leave this unresolved until I get confirmation that at least one
other person has been apply to apply the fix patch and get everything to work.

The remaining documentation tasks will get completed as time permits.

Reid.

> The Stacker compilation errors you described only occurred *after* I did a cvs
> update at the top level. Stacker programs used to compile just fine, before the
> update. 

This was because I added the assertion to the verifier to ensure that you
couldn't construct illegal globals!  :)

Brian, can you check in the patches after the "1.0" patch when you get a chance?
 I'm not familiar with how the autoconf stuff is set up.

Thanks all,

-Chris

The patches are in. I disabled Stacker/{test,samples} because they currently
assume Stacker is in your path. Feel free to mark this RESOLVED FIXED as you see
fit.

Created an attachment (id=35) [details]
Documentation Update For Stacker

This update now contains additional details about my experience developing
Stacker using LLVM. I've highlighted the salient points I think might get
missed easily or that I found saved me time. 

This document is pretty much done except for the "internals" section at the end
for which it just references the code. Perhaps someday I'll write more about
this.

Applied, thanks.

Created an attachment (id=37) [details]
Another Documentation Update

Why does getting documentation right always take longer than the actual thing
being documented ? :)

This patch cleans up several typos and adds a "to do" list.

I attempted to apply your "Another Documentation Update" patch, but it didn't 
go in cleanly.  I'm not sure exactly what's up here, so I'm not checking it in.

-Chris

That's strange. I just tested the patch again (like I always do) against the
latest CVS version. It worked fine and produced the same source file (zero diffs).

Did you: 
1. apply "Documentation Update For Stacker" before "Another Documentation
   Update"?
2. cvs update
3. Apply it from the root? (i.e. cd $SRC_ROOT ; patch < doc_patch.txt) ? I
   alwaysgenerate patches relative to the root.

Do you want to me post the patch again in case it got corrupted in transit?

Reid.

This is what I get, from CVS.  The previous patch was applied by Brian (this I 
verified):

~/llvm$ patch -p0 < doc.patch
patching file docs/Stacker.html
Hunk #1 FAILED at 6.
Hunk #2 succeeded at 108 (offset 24 lines).
Hunk #3 FAILED at 132.
Hunk #4 succeeded at 626 (offset 235 lines).
Hunk #5 succeeded at 424 (offset 24 lines).
Hunk #6 succeeded at 681 (offset 235 lines).
Hunk #7 FAILED at 708.
Hunk #8 succeeded at 521 (offset 24 lines).
Hunk #9 succeeded at 1040 (offset 235 lines).
Hunk #10 FAILED at 1219.
Hunk #11 FAILED at 1467.
5 out of 11 hunks FAILED -- saving rejects to file docs/Stacker.html.rej

Also, this:
+     BinaryOperator* or1 = 
+       new BinaryOperator::create( Instruction::Or, a, b, "", tail );
+

is buggy: it should not have the 'new' there.  :)

-Chris

Created an attachment (id=39) [details]
Another Documentation Patch (Round 2)!

Here's another shot at this patch. I tested this patch with the following,
please do the same:

bash-2.05b$ mv Stacker.html Stacker.html.save
bash-2.05b$ cvs update Stacker.html
U Stacker.html
bash-2.05b$ patch -p0 < stacker_doc.txt
patching file Stacker.html
bash-2.05b$ diff Stacker.html Stacker.html.save
bash-2.05b$

The same >should< work for you. 

That worked, but not from the top level.  :)

Applied. 

-Chris

Oops, forgot to copy my "cd" command in the last post. I wanted to ensure there
wasn't a directory issue.  The original patch must have been corrupted in transit.

Reid

Stacker has been applied to mainline, so I think this PR is done for now.  Any
stacker bugs or enhancements should be filed as new PRs.

-Chris

Created an attachment (id=43) [details]
Patch to fix Makefile in tests directory - remove dependency on users
environment.

This patch fixes projects/Stacker/test/Makefile to remove dependencies on the
user's environment for running the test programs. Previously the stckrc
(Stacker Compiler) needed to be in the user's PATH and the stkr_runtime.o
(Stacker Runtime Library) needed to be in the user's LD_LIBRARY_PATH. These
dependencies have been removed and encoded into the Makefile. Additionally, the
Makefile will now use the corret version (based on the CONFIGURATION Makefile
variable) of the executables for building the test programs.

Created an attachment (id=44) [details]
A patch to fix Makefile in projects/Stacker/samples

This patch provides much the same functionality of the Makefile in the test
directory. It removes dependence on the user's PATH for building the sample
programs.

These two patches are a definite improvement, and have been applied, but they
can still use a bit of work.

In particular, it would be good to make better use of the things defined in
Makefile.rules.  For example, you could define LLC_EXEC as:

LLC_EXEC = $(LLVMTOOLCURRENT)/llc

Better yet, all of the variables that define the locations of tools could be
moved up into the top-level Makefile.rules file, which already includes settings
for LLVMAS, BURG, TBLGEN and LGCCLDPROG (it would also be nice if these all had
uniform names, like <foo>_EXEC, but that's another story :)

Also, instead of hardcoding in the use of gcc, it would probably be better to
use $(CC).

Overall, I'd like to see stacker become more of a first class citizen in the
LLVM tree  :)

-Chris

Created an attachment (id=50) [details]
Minor edits and corrections to the documentation

In this patch I fixed some of the example code that wouldn't compile, corrected
some typos, added a "to do" item, and enhanced the look of the table of Stacker
definitions.

Committed, thanks!

-Chris