Calculating overall data file fragmentation...

David Adams (4/17/14 11:07AM)
J?rg Knebel (4/17/14 3:15PM)
J?rg Knebel (4/17/14 5:16PM)
Tim Nevels (4/17/14 6:01PM)

David Adams (4/17/14 11:07 AM)

<CAPXPcQv6XGv4eW9FtPMjrmmEzg4bAogkC3Cewngi+Cx3vpiRXg@...

The Get table fragmentation is handy for checking the data file but
now I'm
hoping to find something comparable to MSC's overall data file
fragmentation result. Checking individual tables, I can find items that
have 100% fragmentation. But is that because 10 records were cleared?
10,000? 1,000,000? There's really no way to know. If a tiny tables has
been
cleared, the % of fragmentation in the total data file may still be
low -
but the table can show a very high fragmentation percentage.

Any suggestions on how to come up with a smarter guess?

Thanks.
J?rg Knebel (4/17/14 3:15 PM)

Hi David,

I think we have to do ourselves but we need much more information as
available.
If memory serves there are 2 notes in the 4D knowledgebase regarding
the address table one of them has "128 bit" in the subject.

On 17/04/2014, at 11:07 AEST, David Adams <dpadams@... wrote:

color><param>00000,0000,DDEE/param>CChecking individual tables, I can
find items that
have 100% fragmentation. But is that because 10 records were cleared?
/color>
Ha, I can top that! I have a table with 100% fragmentation that
had/has only one record for all the time of its existence.

color><param>00000,0000,DDEE/param>AAny suggestions on how to come up
with a smarter guess?
/color>
After you checked the fragmentation of all your tables you could use
TRUNCATE TABLE after you secured the valid records into a temp-table
(check the knowledgebase again) to clean the address table and Compact
data file afterwards.
I just did this in a v12 database and MSC looks good after that stunt.
:o))

You could add something like Dump Indexes / Rebuild Indexes.

BUT,
if you want more detailed information before you do anything, look for
a demo JPR showed in Sydney Aeons ago (4D v6.-something) containing
detailed data analysis.
The source code was never available unfortunately, but maybe JPR could
give some hints?=80¶

Another way would be a DIY-project (just a thought):

- get the fragmentation info per table (Get table fragmentation)
- get total numbers of (non deleted) records per table
- get the real size of each record of each table by looping
trough
all the fields of a table:
$RecordSize := $RecordSize + OverheadSize(FieldType) +
Size(Field)
- get the size of each table
$TableSize ¬=A0:= $TableSize + $RecordSize
- get the size of all tables
¬=A0¬=A0¬=A0¬=A0¬=A0¬=A0¬=A0$DataSize :=
$DataSize + $TableSize
- get the size of the datafile on the HD
¬=A0¬=A0¬=A0¬=A0¬=A0¬=A0¬=A0Get document size

With all this numbers you could do something. ;-)

HTH

Regards
J?=B6rg Knebel, M.Eng. - 4D Developer since 1991
TTT Data Systems Pty Ltd
Phone: +61 (0)2 6334 4730
www.tttdatasystems.com.au
J?rg Knebel (4/17/14 5:16 PM)

Hi Bob,

On 17/04/2014, at 15:30 AEST, Bob McKeever <bobmckeever@...
wrote:

color><param>00000,0000,DDEE/param>IIt will be more accurate to use
API Pack --> Record to Blob, and look at the blob size to get the real
size of a record.
/color>
Thanks for this input.
Definitely a step I'll consider.

Thanks.

Regards
J^rg Knebel, M.Eng. - 4D Developer since 1991
TTT Data Systems Pty Ltd
Phone: +61 (0)2 6334 4730
www.tttdatasystems.com.au
Tim Nevels (4/17/14 6:01 PM)

On Apr 17, 2014, at 3:50 PM, JPR wrote:

color><param>00000,0000,DDEE/param>TThe Data Fragmentation has almost
always been a concern for 4D Developers. I perfectly remember the
experiences I've shown during trainings, but it was with 4D V6.5 and
2004. I made DataAnalyzer in 1996, just to verify the impact of data
fragmentation on response time. Things have changed a lot since.
/color>

I remember DataAnalyzer. That was the most amazing pure 4D application
that I had ever seen at that time.

Using only 4D code -- no special plugins needed -- it would open a 4D
data file and read it byte by byte and analyze it. Then show the
layout of the indexes and records in the data file in graphs and
pictures. Record sizes, index sizes, fragmentation, etc. The pictures
-- in Mac OS PICT format -- were built pixel by pixel with pure 4D
code. JPR showed the awesome power of the BLOB TO PICTURE ?command.

I had not thought about DataAnalyzer in a long time. ?Great tool JPR!

color><param>00000,0000,DDEE/param>TThe techniques I've given during
V6.5 trainings (like 'Stable data' technique) are more or less
obsolete on V14. You can't apply steam engine techniques on turbine
optimization&Ouml;
/color>
Well put, well put. Another JPRism worth repeating. :)

Are you going to do another session -- or at least thinking about
doing another session -- at this year's 4D Summit in Texas?

Tim
Reply to this message

Summary created 4/18/14 at 6:15PM by Intellex Corporation

Comments welcome at: feedback@intellexcorp.com