Forum - MCS Electronics

 

FAQFAQ SearchSearch RegisterRegister Log inLog in

5 k lookup table
Goto page 1, 2  Next
 
Post new topic   Reply to topic    www.mcselec.com Forum Index -> BASCOM-AVR
View previous topic :: View next topic  
Author Message
autoguider

Bascom Member



Joined: 24 Sep 2007
Posts: 74
Location: Aachen

germany.gif
PostPosted: Mon Jan 11, 2021 6:16 pm    Post subject: 5 k lookup table Reply with quote

Dear All,
for a simple solution of a complex positioning issue I am considering to use a lookup table.
It is a 2 dimensional matrix of appr. 50 x 70 word-elements ( 7-8 kbytes) to be stored in an EEPROM
My preferred idea is to implement it as one array.
I fear that it is not possible.

The ATMEGA 128 (DIP 40, what I prefer) )has 4kB of EEPROM, which is not sufficent. Perhaps I may come along with some interpolation.
Using an external EEPROM like AT24Cxx is also a possibility.
I have no experience with external memories yet.
In the samples I˛c there is an example how to access this EEPROM. Then calculation of the address will be a challenge for me.

To me it is not clear how to fill the initial values into the EEPROM.

If you have some advise how to create and use a lookup table of this size please be so kind to let me know.

best regards
Christian




(BASCOM-AVR version : 2.0.8.3 )
Back to top
View user's profile
MWS

Bascom Member



Joined: 22 Aug 2009
Posts: 1864

blank.gif
PostPosted: Mon Jan 11, 2021 8:03 pm    Post subject: Reply with quote

The obvious solutions:
- reduce the array through interpolation to fit the supporting points into 4k
- use Flash instead of EEProm
- create a formula to approximate the array
- use external EEProm
The limited information you gave does not allow to tell what's better.
Quote:
To me it is not clear how to fill the initial values into the EEPROM.

You need a helper-function within the ATM128, which receives the values via UART and writes them to external EEProm.
To transmit data use Bascom's terminal emulator, then 'Terminal -> Send File', other terminal programs like HTerm work similar for this purpose.
Back to top
View user's profile
O-Family

Bascom Expert



Joined: 23 May 2010
Posts: 208
Location: Japan

japan.gif
PostPosted: Tue Jan 12, 2021 1:16 am    Post subject: Reply with quote

What format is the data you want to store?
If it is byte or word data or a character string, it is easy to look up by using the [DATA] instruction.
In addition, if it is a binary format or a data file, it can be incorporated with the [$INC] instruction and operated with the [RESTORE] instruction and [READ] instruction.
Back to top
View user's profile Visit poster's website
JC

Bascom Member



Joined: 15 Dec 2007
Posts: 495
Location: Cleveland, OH

usa.gif
PostPosted: Tue Jan 12, 2021 3:02 am    Post subject: Reply with quote

Some good ideas suggested above.

Perhaps ordering a couple of EEPROM chips and working with them, as a sub-project, would be the “best” approach.
If you order some, be careful in the chip’s description! Many ore listed as XYZ size in BITS, not Bytes, so be careful.

An option I don’t see mentioned above would be using a form of data compression for the table. If the data elements in at least one part of the 2-dimentional array increase linearly, (e.g. 0, 1, 2, 3, 4, …, 49, 50), then the upper byte of the two byte Word data doesn’t change from one value to the next with every entry. You have a long string of entries with the same upper byte data value in a row, as the lower byte increases. You could, therefore, “compress” the data. Don’t store the upper byte with every lower byte.
The lookup algorithm would have to be a little bit smarter to understand the compressed data format, instead of the trivial, expected format.
How much compression, and byte savings you get, depends upon how the data set looks.

Storing, for example, for one of the array’s variables:
0, 0
0, 1
0, 2
0, 3

0, 253
0, 254
0, 255
1, 0
1, 2
1, 3

Uses a lot of space.
If you stored the first Word for a block of data as the full word, using two bytes, and then just stored the lower byte until the upper bye changed, you would save a lot of room.
The 2nd variable of the 2-Dim array would likely require the use of both Bytes for the value.
Still, you could almost obtain a 25% reduction in the storage space, if the data set makes this approach useable.

Regarding writing the original data to the EEPROM ( s ):
Will the EEPROMs be soldered to the final project’s PCB, or in a socket?
If in a socket, then one could put them in a breadboard, with a Nano, (with its USB interface), and use a PC to download the data to the Nano, (or other micro), which then loads the data into the EEPROMS.

When you write this software be sure to also write a subroutine to read back the EEPROM’s data to verify that it was all written as you intended.
One might verify the data Byte-by-Byte, after writing the full chip, or page-by-page, as it is written.

If the EEPROMs will be soldered into the final project’s PCB, then remember that the board could have TWO programs, one which downloads and stores the EEPROM data, and then once that is done a second program is downloaded to the micro to actually do what the project is suppose to do. Make sure you build in an interface for downloading the data for the first program to use!

If the data is calculatable, then obviously one could have the first program just calculate the data array and store it.

If that is the case, however, then one could likely just calculate the data as needed within the program. If the uC isn’t fast enough to calculate the data as needed, then switch to an XMega. They run at 32 MHz instead of 16 or 20 MHz, in spec, so one gets a LOT more processing power per mSec.

Good luck with your project!

JC
Back to top
View user's profile Visit poster's website
MWS

Bascom Member



Joined: 22 Aug 2009
Posts: 1864

blank.gif
PostPosted: Tue Jan 12, 2021 8:39 am    Post subject: Reply with quote

O-Family wrote:
What format is the data you want to store?
If it is byte or word data or a character string, it is easy to look up by using the [DATA] instruction.

If it would be byte, then the question would not exist.
autoguider wrote:
of appr. 50 x 70 word-elements

Quote:
In addition, if it is a binary format or a data file, it can be incorporated with the [$INC] instruction and operated with the [RESTORE] instruction and [READ] instruction.

Restore() and Read() would be for sequential access, which is a bad equivalent to a random accessible array.
You've surely wanted to suggest Lookup().
Back to top
View user's profile
MWS

Bascom Member



Joined: 22 Aug 2009
Posts: 1864

blank.gif
PostPosted: Tue Jan 12, 2021 9:33 am    Post subject: Reply with quote

JC wrote:
An option I don’t see mentioned above would be using a form of data compression for the table.

For some good reason.

In the world of PCs you have tons of processing power and beside audio/video streaming mostly no need for real-time processing.
Where it is about audio/video, the 'tons of processing power' apply.
The disadvantages of compression is that random access to the array on the lowest level is not possible and if encapsulated by a function, it will be slow while having lots of overhead.
Quote:
You could, therefore, “compress” the data. Don’t store the upper byte with every lower byte.

For the type of compression you've suggested, you need a marker, which identifies the real word from the compressed word, the marker takes space too, thus it depends on the compressibility of the data, whether you have a gain.
In case accessing time is not the problem, and the difference between consecutive values is smaller than 128, I'd suggest an approach like DPCM with one array-dimension's amount of supporting points.
As compression rate depends on the data set to compress, it may happen that one data set fits the AVR's 4k integrated EEProm, while a different data set won't.
What then, trash everything and start over again?
Back to top
View user's profile
hgrueneis

Bascom Member



Joined: 04 Apr 2009
Posts: 880
Location: A-4786 Brunnenthal

austria.gif
PostPosted: Tue Jan 12, 2021 9:50 am    Post subject: Reply with quote

Ferroelectric Nonvolatile RAM are available in many memory sizes.
Most are with fast I2C (TWI) or SPI (some up to 40 MHz).
Advantage: no wait for writes and decades of data retention and close to unlimited writes.
Regular eeprom are very limited in write cycles and slow.
Flash memory needs a lot of control.

Regards
Hubert
Back to top
View user's profile
autoguider

Bascom Member



Joined: 24 Sep 2007
Posts: 74
Location: Aachen

germany.gif
PostPosted: Tue Jan 12, 2021 2:18 pm    Post subject: Reply with quote

I want to control the position of the slit in the dome of my small astronomical observatory. The problem is that there are 2 telescopes based on an equatorial mount. They have a appr. 40 cm offset from the central position. Both telescopes must have a free view through the slit. Distance between the optical axis is 40 cm. Which is another offset. The slit has a free opening of 60 cm. So the offsets matter.
Doing a calculation considering just one telescope is a bit complicated but not rocket science. Having to consider the offset of the second telescope makes it difficult. The optics does not look perpendicular through the slit but sometimes tilted.
So the idea is to measure the position and put the data in a lookup table.
For the horizontal position of the slit I intend to go with a resolution of 1 degree. This means around 360 different positions in azimuth. This I cannot cover with the datatype BYTE but with WORD.
As I know how the movement is going to be for the next e.g. 8 hours I can load only the required values from the EEPROM. Speed is not an issue.
Measuring will be a time consuming work. Also putting the data into a EEPROM is a challenge. So reading the data from an ASCII file via a loading program is a good idea.

BR
Christian
Back to top
View user's profile
MWS

Bascom Member



Joined: 22 Aug 2009
Posts: 1864

blank.gif
PostPosted: Tue Jan 12, 2021 4:23 pm    Post subject: Reply with quote

autoguider wrote:
For the horizontal position of the slit I intend to go with a resolution of 1 degree. This means around 360 different positions in azimuth. This I cannot cover with the datatype BYTE but with WORD.

Maybe use what you have, if 1 degree does not fit into a byte, then 1.40625 degree would, would that be too coarse?
Also you waste a lot of memory if you you use only 9 bits out of 16 bits. If the process is slow, then you can consider a byte array out of 4096 bytes also as a bit array of 32768 bits.
If every entry takes 9 bits, you have 3640 entries, which is enough for
Quote:
It is a 2 dimensional matrix of appr. 50 x 70 word-elements

It will take a few calculations to virtually build up a 9 bit data matrix within a real 4096 byte linear memory range.
But nothing complicated and on the other hand, calculations are what these controllers are made for.

Mind to explain:
Quote:
50 x 70 elements


It may as well be, that you take the wrong approach, maybe a lookdown would be preferable to a lookup.

What are the control values and how the output should look like?
In the moment [50] and [70] are input, output is degree.
What is [50] and what is [70]?
Quote:
So reading the data from an ASCII file via a loading program is a good idea.

That's how a bootloader works.
Back to top
View user's profile
hgrueneis

Bascom Member



Joined: 04 Apr 2009
Posts: 880
Location: A-4786 Brunnenthal

austria.gif
PostPosted: Tue Jan 12, 2021 10:40 pm    Post subject: Reply with quote

if it was me, I would use a digital compass and degree sensors(or pots with calculation), to control everything with steppers or servos.

Regards
Hubert
Back to top
View user's profile
hgrueneis

Bascom Member



Joined: 04 Apr 2009
Posts: 880
Location: A-4786 Brunnenthal

austria.gif
PostPosted: Tue Jan 12, 2021 11:04 pm    Post subject: Reply with quote

It would be fairly easy to do bar strip for all the units and have it more exact than one degree. I would rather implement a sensor based system. Then you could control the opening also depending on the use of telescope1 or 2 or both. Advantage: No map, just coordinates.
And you can get that from the Stellarium program. And if you use only one scope, then you have a narrower opening which will give you less light pollution..

Regards
Hubert
Back to top
View user's profile
autoguider

Bascom Member



Joined: 24 Sep 2007
Posts: 74
Location: Aachen

germany.gif
PostPosted: Wed Jan 13, 2021 12:00 am    Post subject: Reply with quote

Hubert,
you got it absolutely right.These are the other options.
Digital compass has to be very much tilt compensated. The telescope tube may point to the zenith. This is a 90° tilt vs. the horizontal plane.
I will also follow this way as an alternative.
I am also looking for e.g. ultrasonic sensors to detect the slit.
The challenge comes because I have to cope with 2 optics and 2 offsets and I must not put the sensors in the opening of the telescope. They have to be on the tube and change their position.
Perhaps matlab or another software may be useful to model it.
Regretfully I have no experience with such a software.

I think reduction in angular resolution so that I can use bytes will help. With bytes the resolution is 1.4 degrees accuracy. This will be sufficient.
I did some modeling just for 1 telescope and for stepper motors. The formulae are not that complicated but you need to calculate with ellipses and not just with circles.
The lookup table would be just brute force.
Sensing would be my preferred solution.

Using a sensor for magnetic fields requires to check and compensate influences of steel and motors in the proximity. And I have a steel pier and at least 2 motors with appr. 2 A each.
Maybe ultrasonic sensors or radar sensors are the solution.

It will become a nice challenging project.

best regards
Christian
Back to top
View user's profile
MWS

Bascom Member



Joined: 22 Aug 2009
Posts: 1864

blank.gif
PostPosted: Wed Jan 13, 2021 12:38 am    Post subject: Reply with quote

autoguider wrote:
I think reduction in angular resolution so that I can use bytes will help. With bytes the resolution is 1.4 degrees accuracy.

Good to hear the straightforward approach works.
I found the the more complicated way more interesting.
Code shows only one way of the conversion, the other would be to read the 9 bit back, should not be difficult. Very Happy
Memory usage is 3938 bytes for an array of 3500 * 9 bit.
Code:
'$Regfile="m128def.dat"
$Regfile="m2560def.dat"                                          ' m2560 to simulate in bigger SRam

$Crystal=16000000
$hwstack=64
$swstack=64
$framesize=64

Config Base = 0                                                  ' array index starts at 0
                                                                 ' one array dimension
Const X_size = 50                                                ' the other
Const Y_size = 70
Const Var_Bits = 9                                               ' dataset of 9 bits
Const Byte_lin_array_size = ((X_size * Y_size * Var_Bits) / 8) + 1  ' calculate space for array
Const 9Bit_Mask_Preset = 2 ^ Var_Bits - 1                        ' mask prototype

Dim Byte_lin_array(Byte_lin_array_size) As Byte                  ' create linear byte array for 9bit datasets
Dim X As Word                                                    ' X-dimension 0..19
Dim Y As Word

Dim Lin_Array_Word As Word
Dim Lin_Array_LB As Byte At Lin_Array_Word + 0 Overlay           ' for low byte access of Lin_Array_Word
Dim Lin_Array_HB As Byte At Lin_Array_Word + 1 Overlay           ' same for high byte

Dim 9Bit_Var As Word                                             ' variable to hold 9 bits
Dim 9Bit_Mask As Word
Dim 9Bit_Inv_Mask As Word

Dim 9Bit_Var_lin_idx As Word
Dim 9Bit_Var_Bits As Word
Dim Byte_lin_Int As Word
Dim Byte_lin_idx0 As Word
Dim Byte_lin_idx1 As Word
Dim Shift_Count As Byte

X = 1                                                            ' set address of dataset
Y = 1                                                            ' same for other dimension
9Bit_Var = &b111000011                                           ' data to write, for tests
'9Bit_Var = &b111111111

9Bit_Var_lin_idx = X * Y_size                                    ' get dataset index
9Bit_Var_lin_idx = 9Bit_Var_lin_idx + Y                          ' same
9Bit_Var_Bits = 9Bit_Var_lin_idx * Var_Bits                       ' calculate absolute bits to address

Byte_lin_idx0 = 9Bit_Var_Bits / 8                                 ' integer division by 8 to get first index in byte array
Byte_lin_idx1 = Byte_lin_idx0 + 1                                ' second index follows

Byte_lin_Int = Byte_lin_idx0 * 8                                 ' multiply again by 8 and
Shift_Count = 9Bit_Var_Bits - Byte_lin_Int                        ' together build the modulo, which equals number of bit shifts

9Bit_Mask = 9Bit_Mask_Preset                                     ' load default mask
Shift 9Bit_Mask, Left, Shift_Count                               ' shift to proper position
9Bit_Inv_Mask = Not 9Bit_Mask                                    ' create bit-delete mask

9Bit_Var = 9Bit_Var And 9Bit_Mask_Preset                         ' clip any input over 9 bits
Shift 9Bit_Var, Left, Shift_Count                                ' shift to right place

Lin_Array_LB = Byte_lin_array(Byte_lin_idx0)                     ' get existing values from byte array
Lin_Array_HB = Byte_lin_array(Byte_lin_idx1)                     '
Lin_Array_Word = Lin_Array_Word And 9Bit_Inv_Mask                ' clear 9 bits for new data
Lin_Array_Word = Lin_Array_Word Or 9Bit_Var                      ' squeeze 9 bits in
Byte_lin_array(Byte_lin_idx0) = Lin_Array_LB                     ' write back to limear byte array
Byte_lin_array(Byte_lin_idx1) = Lin_Array_HB                     '

End
Back to top
View user's profile
AdrianJ

Bascom Expert



Joined: 16 Jan 2006
Posts: 2477
Location: Queensland

australia.gif
PostPosted: Wed Jan 13, 2021 11:41 pm    Post subject: Reply with quote

Since its essentially a geometric problem, I would solve it by doing some analytic geometry to derive actual math expressions for the values you are trying to find, then just evaluate the formula in real time anytime you want a result. You sre going to need that calculation anyway to create your data array, why not just do it in real time ? Might take some work to implement the math in Bascom, but its far more flexible in case of a change to the geometry of the system.
_________________
Adrian Jansen
Computer language is a framework for creativity
Back to top
View user's profile Visit poster's website
O-Family

Bascom Expert



Joined: 23 May 2010
Posts: 208
Location: Japan

japan.gif
PostPosted: Thu Jan 14, 2021 12:39 am    Post subject: Reply with quote

Rather than forcing a weak microcomputer to perform advanced calculations, it is possible to have a powerful personal computer perform calculations, or output simple calculation results such as Excel to a text file and incorporate it into the microcomputer.
I think that may be more accurate and easier.
I hope there will be many suggestions that match the level of the creator.
Back to top
View user's profile Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.mcselec.com Forum Index -> BASCOM-AVR All times are GMT + 1 Hour
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum