Plugin - Running C
- Restrictions
- How to write a Plugin
- Automation (updated - new program from Peter (G8JCF))
- How it works
- Plugin2
Special note for version 2.3, the defined constants have been altered slightly and so if using version 2.3 substitute call(?plugin,nnn) for call(?plugin(0),nnn).
A plugin is a machine code function that will enable something to by done that could not normally be done by using ByPic alone. This is probably required due to speed of execution. Plugin's have been introduced at release 2.07-77, releases prior to this do not have the call() function and so cannot be used to run plugin's. In a nutshell it allows a function coded in C to be called from ByPic.
First I will show you what a plugin is and then explain how to make one.
http://www.byvac.com/mBlib/flb/Tutorial/plugin/plugin1_2.bas
// plug in example constant test1_pi { 0x27bdffe8, 0xafbe0014, 0x03a0f021, 0xafc40018, \ 0x8fc20018, 0xafc20000, 0x8fc20000, 0x8c420000, 0xafc20004, 0x8fc20000, \ 0x24420004, 0x8c420000, 0xafc20008, 0x8fc30004, 0x8fc20008, 0x70621002, \ 0x03c0e821, 0x8fbe0014, 0x27bd0018, 0x03e00008, 0x00000000 } // constant test2_pi { 0x27bdffe8, 0xafbe0014, 0x03a0f021, 0xafc40018, \ 0x8fc20018, 0xafc20000, 0x8fc20000, 0x8c420000, 0xafc20004, 0x8fc20000, \ 0x24420004, 0x8c420000, 0xafc20008, 0x8fc30004, 0x8fc20008, 0x00621021, \ 0x03c0e821, 0x8fbe0014, 0x27bd0018, 0x03e00008, 0x00000000 }
The above is actually two plugin's, the first one called "test1_pi" takes two integers and returns the result of them both multiplied together and the second one called "test2_pi" takes two integers and adds them together. There are two simply to show that plugin's can be setup and used at any time.
http://www.byvac.com/mBlib/flb/Tutorial/plugin/plugin_example.bas
// Example of how to use a plug in // Verison 2.07 77 onwards // Load plugin1_2.bas first and then flsave("") // function test() dim ab(2), x ab(0) = 77 ab(1) = 88 x = call(?test1_pi(0),?ab(0)) print "\nresult of test1_pi is ",x x = call(?test2_pi(0),?ab(0)) print "\nresult of test2_pi is ",x endf
This is the program that runs the plugin's. As can be seen they are run with the call() function that will be explained later. First just to confirm that it works, install the plugin's and run them as follows. To try this do the following carefully:
- load the actual plugin's, these are the constants in the first file.
- type flsave("")
- load the plugin_example.bas (the second one and then type test()
This will work for both MX1 and MX3 type PIC32's. The above is trivial but illustrates how to use and run a plugin. Often the end user may only need to do this as the plugin may have been written by somebody else. As you can see the plugin is defined by the constant statement. It places runnable code first into RAM and then, by the use of flsave("") into the Flash. Code (native machine code) cannot be run from RAM and so all plugin constant definitions must be saved to Flash before use.
Some Restrictions
A plugin must be a single function and must not call other functions although several plugin functions can exists and be called by the call() ByPic function. The plugin will always return an integer and always expect a character pointer as a parameter. This really should be a void pointer but I am old fashioned. The reason for this restriction is explained later. It has been implemented this way so that a plugin is easy to use and most of the time a single function is all that is needed.
How to write a plugin
A plugin is written using a C compiler for the target processor, in this case it is the PIC32 (gcc) compiler. It has to be a C compiler or one that will use the same call and return conventions. In this instance we are using the PIC32 and the PIC32 C compiler provided free from Microchip. We can also use MPLAB IDE and the resource file contains a project for this example.
It is done is steps which can be automated but have not been for this example to make it as clear as possible.
Step 1 Writing the function/s
int test1_pi(char *c) { int *ip, a, b; ip = (int*)c; // convenent integer pointer a = *(ip); // contents of memory as integer b = *(ip+1); // contents of memory as integer return (a * b); }
This is the C function that generated the machine code used for the constant above. The function must always be defined as int <function name> (char *). This is so that it matches the call() function when calling the function. By using a character pointer we can pass back and forth any values we like, the minor inconvenience of course is that we need to cast the memory address into what we need. In the above example, in ByPic an array of integer is defined: dim ab(2) and an integer value is placed into each one of the two defined integers, ab(0) and ab(1). What this does is place the two integers in consecutive memory locations, the starting address is ?ab(0), this is passed to the C function so that ip will now point to the start of the first integer, it is just then a matter of retrieving those integers. If your pointer arithmetic and casting are a little bit rusty then here are some more examples:
Byte values:
int plugin (char *c) { char *bp,a,b; // read byte values a = *(bp); b = *(bp+1); // set byte values *(bp) = 122; *(bp+1) = 10; ...... }
It should even be possible to create a structure to handle the values but lets not get carried away. Using a memory address offers no restriction as to what can be sent to the plugin and what the plugin can send back. Just a reminder, a plugin cannot call any other function, even library functions and so something like strcpy(bp,"fred") is out of the question. An explanation as to why this is so is at the end of this text.
Step 2 Compiling
It is most important to use some compiler switches when compiling the functions. This is to make the code relocatable, as when it is installed as a constant, it could be anywhere in memory. The switches are:
-mlong-calls -fPIC -mno-gpopt -mshared -mno-abicalls
Although I am not sure about the '-fPIC' switch as it does give a warning message. When using MPLAB, these are set in the build options dialog that is invoked when right clicking on the source file.
The source file must also contain 'main()' but for our purposes this is just a blank function and can be ignored. When built using the C compiler an '.elf' file is created, in the example in the resource file (the zip at the start of the text) the plugin.c source file gives a file called plugin.elf. This is a binary file that contains everything about the code just compiled, what we are after is a disassembly of our function along with the machine code for it.
Step 3 Create a dissasebly file called dis.txt
To do this open a dos box:
If you get an error at this point it is because the microchip\<compiler location>\bin is not on the path, you will need to locate the program "pic32-objhump.exe" which will be in "program files" in a microchip directory, the easiest way to deal with it is to copy it to the same location as the elf file and then run the command line again.
All being well this will produce a file called dis.txt, open it with a text editor and locate the function you need, in the example this will be test1_pi. The bit of the file we are interested in is the actual machine code that makes up this function. This has been created by the linker for us:
This is an extract of the file, all we need do now is to place those numbers in a constant statement as shown at the top of this text. In fact if you look at the numbers highlighted in blue they will match with the constant statement at the top of this text, of course when using the constant statement a prefix of 0x is required. This can be done manually or with the help of a text editor.
That is all the steps that are needed in order to create a plugin.
Automation
You may also be interested in installing this program from Peter (G8JCF) rather than using the manual process below. This can be downloaded from his site. It makes the automation much simpler. As a bonus there is also included some further tutorials and information that will be installed along with the program.
Thanks Peter for this great contribution!
It is a bit laborious to create the constant file from the text on the dis.txt file, particularly as it is not likely to work first time. To help with this the resource file has a Python program that will automate the process of creating the dis.txt file, extracting the machine code and creating the plugin.bas file. It is called 'distobas4.py' and will need Python installing in order to run it. A simple C or Delphi program could be written to do the same thing, if you write one let me know and I will post it on this site. I am sticking with Python for now as it is my new friend.
To create a .bas file with the code from the elf file open a dos box (cmd prompt) and type the following:
python distobas4.py dis.txt test1_pi
This assumes that your C function is called 'test1_pi', if not substitute the name of your own function. There is a text file called create.cmd in the resources zip file at the beginning of this text that was used to create the two plugin files, they were combined into one file using a text editor to create plugin1_2.bas
How it all Works and Why
First of all a bit of history, this is not the first plugin arrangement I have created. The first one was for the original BV_Basic. That one I instinctively felt was too complicated to implement to make much progress on, although it was implemented and some worked examples given. The main drawback was that it had to be linked to a specific area of memory and so flash had to be reserved for that purpose. It also had to go into the exact location which meant finding out where it was going and then setting up the linker file to link to that location. The next problem was getting it from the PC to the Flash, this used a binary load system. This was okay for the expert developer but the plugin was really intended for a casual user and to be used by somebody who had not written the plugin but just wanted to use it. The complexity made this unworkable.
For this plugin I wanted it to be as simple as possible, ideally just a file at the top of the ByPic program. The full intention being that a plugin could be written for anybody else to use, the emphasis being on the simplicity of use even at the expense of more complex creation. It has been possible to nearly achieve that with some restrictions. A crucial part of this type of implementation is that relocatable code is required so that the constant can be defined at any time in any program.
Relocation, Relocation, Relocation
The compiler will happily generate relocatable code for the MIPS (PIC32) processor and produce an assembly output with relative rather than absolute branches to other parts of the code. Relocatable means that the function can be placed anywhere in memory and still run. The trick to doing this is to not specify any absolute addresses, if an address is required then it is specified relative to the current location. As an example I could live in any street and my neighbour is 1 door down (or up). How ever if I lived at number 6 and said my neighbour lived at number 7 then it would only apply to that particular street and house number.
When creating a program form source code to machine code it is nearly always a two step process, first the compiler and then the linker. The two stages were separated out some time ago so that code could in fact be placed anywhere in memory, the linkers job is to take several pieces of code from all sorts of places and link them together so that they will run on the target machine and here is the problem for my scheme. It is the linker that decides on the absolute memory locations (something has to) and so by the time the linker has done its job the memory locations have been allocated. There is a file called the linker script that decides where everything should go.
The ideal method of course is the DLL (although not the Windows implementation apparently) however we are dealing with a microcontroller with limited resources, having said that it may still be possible to do it via this route using the host PC's resources.
Back to the problem in hand: Provided the code, not the assembler code but the actual machine code is relocatable then the scheme works well. A constant can be defined at any time to hold the code and it will run in any location and in any program, this gives a massive amount of flexibility. The only downside is that it must be one function and that function must not call other functions. So why?
As far as I am aware the MIPS instruction set does not contain a relative call, it has relative branch that we are taking advantage of using the compiler switches but when it comes to a function call the JAL or LALR instruction is used and this requires an absolute address to either follow the instruction or to be placed into the register, all taken care of by the linker of course. If the plugin function were to call another function then it would branch to an absolute address that would definitely not be the correct address and so would not work. There are some ways round this that I can think of.
Assembler
Instead of using C, use MIPS assembler and make sure all of the code is relocatable, when calling another function a branch could be used and the link address saved to a register manually. The C calling convention must still be observed but this can be easily copied from the code that C produces. A modification of this scheme is to compile the code in C but modify any absolute jumps.
Link to the Correct Address
This is quite possible but very restrictive and is similar to the old scheme (see history above). The method is to find out the address of the constant, by creating a constant and then printing out its address, e.g.
constant temp 1 print hex$(?temp)
Now use this address in the linkers '.ld' file. You can probably ignore all of the nme, general exception type functions but will need to carefully make sure that any address referenced in any of the functions is included in the constant statement. The code generated will be restricted to that release and that set of conditions.
DLL
The goal of ByPic is that it can be used on any host with a terminal, no special tools are needed. The only way for this method to work is to have a specialist tool that will communicate with ByPic find out where the code is going and then link and download the code to that address.
Something Missed
I am assuming that there is no instruction in MIPS for a relative call and / or that the gcc compiler / linker cannot achieve this functionality – I could be wrong.