| OS Version | Package | Tested | Notes | |--------------------+---------+-------------+-------------| | mx linux 21.x | native | yes | | | mx linux 21.x | generic | in-progress | | | Ubuntu 18.04 | native | in-progress | | | Ubuntu 18.04 | generic | in-progress | | | Ubuntu 20+ | generic | no | manual deps | | Debian ?? | generic | no | manual deps | | RedHat ?? | generic | no | manual deps | | Windows ??, CygWin | generic | no | manual deps | | MacOS ??, brew | native | no | manual deps | | MacOS ??, brew | generic | no | manual deps |
If you install with a libre-bib.deb package with a package manager such as "apt", all of the required and most of the optional packages will be installed.
If you are installing from tgz file, then you'll need to install these manually.
Packages can be found at: https://moria.whyayh.com/rel/released/software/own/libre-bib/
"variables_order" to this:
variables_order = "EGPCS"
Run:
bib setup-bib
Fix any errors then run it again, until no more errors.
The DB packages, mariadb-client and mariadb-server, have been installed. Most likely the mariadbd process will already be running. Verify this with:
ps -fC mariadbd
If you don't see it running, you'll need to consult the mariadb docs to get it running.
The first one is a good source for quickly getting going. Depending on your distribution, you may need to do things a bit differently.
Test the connection on the server system
sudo mysql -P 3306 -u root -p
Most likely you'll use your sudo password, or the password you setup for the mysql DB root user.
While signed in as root user to the DB type these commands. Replace the $cgNAME variables with the values of those variables in your project/conf.env file. You can change those now or just use the example names to try things out.
sudo -s mysql -P 3306 -u root -p
create database $cgDbName; show databases;
Create users
The create user and grants are best done with the 'root' DB user on the mysql system.
create user 'admin'@'localhost' identified by 'ADMIN-PASS'; grant all privileges on *.* to 'admin'@localhost; create user '$cgDbUser'@'localhost' identified by 'USER-PASS'; grant all privileges on $cgDbName.* to '$cgDbUser'@localhost; flush privileges; select user from mysql.user; show grants for 'root'@localhost; show grants for 'admin'@localhost; show grants for '$cgDbUser'@localhost; quit;
Test a local connection with $cgDbName
mysql -P 3306 -u $cgDbName -p -h 127.0.0.1 $cgDbName
If you will be using libre-bib on the same system as the DB, then try connecting with the "bib" command.
bib connect
If that doesn't work look at the cgDbHost variable setting in conf.env. It should be set to "localhost" or "127.0.0.1" for local access. Try again, If that works, your conf.env setting are good for continuing.
This shows a minimal setup with a local DB. This will use the example files and the minimal default configuration.
For a detailed example see Appendix "A Full Example."
mkdir -p project/biblio
cd project/biblio
bib setup-bib # This creates your default conf.env file
edit conf.env # Uncomment and set these values
cgDbName="YOUR-DB-NAME"
cgDbUser="YOUR-DB-USER"
cgDbPassHint="YOUR-HINT"
bib setup-bib # Your project are will be setup
bib # List the bib commands
bib connect # Connect to DB to cache the password
bib import-lo # Import the biblio.txt file
bib ref-new # Use DB values for any new REFs
bib ref-update # Update REFs with any DB changes
libreoffice example.odt
# Load styles from another odt file (optional)
# Add bibliography to end of document
# Set the bibliography Type > Brackets to "none"
bib style-update # Define the bibliography > Entries for the different Types
bib style-save # Run this if you change bibliography > Entries
This is a quick summary of the directories and files setup in your project directory. The details will be describe in later sections as they are used.
The bib commands will notice changes and rebuild any dependent files as needed. So you might see more things running than what you've seen before. The "Env-Var" column shows the variable for the File-Dir. The Cmd column shows the command or commands that create or use the File-Dir.
$ cd $HOME | File or Dir | Var / Cmd | |----------------------------+------------------------------------------| | conv.env | Cmd: setuup-bib | | biblio.txt | Var: $cgLoFile; Cmd: setup-bib | | biblio-note.txt | Var: $cgLoFile; Cmd: setup-bib | | key.txt | Cmd: setup-bib | | example.odt | Var: $cgDocFile; Cmd: setup-bib | | status/ | Var: $cgDirStatus; Cmd: setup-bib | | .... import-lo.date | Cmd: import-lo | | .... backup-lo.date | Cmd: backup-lo | | backup/ | Var: $cgDirBackup; Cmd: setup-bib | | .... backup-lo.csv | Var; $cgBackupFile; Cmd: backup-lo | | .... backup-lo.csv.bak | Cmd: backup-lo | | .... backup-lo.csv.bak.~2~ | Cmd: backup-lo | | .... backup-lo.csv.bak.~1~ | Cmd: backup-lo | | tmp/ | Var: $cgDirTmp; Cmd: setup-bib | | .... .pass.tmp | Var: $cgDbPassCache; Cmd: connect | | .... biblio.txt | Var: $cgDirTmp/$cgLoFile; Cmd: export-lo |
File: /opt/libre-bib/doc/example/conf.env - Example document config
This file is copied to $PWD/conf.env when you first run bib.
File: /opt/libre-bib/etc/conf.env - System config
All the default values must be defined in this file. You can edit this file to override things for all your bib directories, but it would be better to edit ~/.config/libre-bib/conf.env. That way the app can be updated without overriding your changes.
File: $PWD/conf.env - Document config
This file is required, but everything can be commented out. Uncomment the ones that are specific to the current bib document.
| Variable | Default | Used by | |-------------+----------------+----------------------| | $cgDbName | biblio_example | Data Base Name | | $cgDbTblLo | lo | import-lo, export-lo | | $cgDbTblBib | bib | import-lo |
Understanding the variables in the conf.env file will probably give you the best understanding of how the libre-bib application works.
The conf.env files are the core configuration files for the libre-bib app. They are executed in this order, so the last definition wins.
1. . /opt/libre-bib/etc/conf.env 2. . ./conf.env
Var: cgDebug=false
If "true" then some diagnostic messages will be output.
Var: cgNoExec=false
If "true" then things will be checked with non-destructive reads. Execution will stopped before anything would be changed.
Note: this is not the same as the "-n" option. "-n" will show the commands that will be executed. cgNoExec forces the command to not make any destructive changes. Files might be copied to backup locations, but tables and files will not be changed.
Var: cgVerbose=true
If "true" the commands being executed will be shown and there could be more status output as things are run.
Note: Currently some errors messages are not output if this is set to "false". If you see no output and no changes, the set this to "true" and try again.
Var: cgDirBackup="backup"
This is the directory name (or path) where backup files are put. "~" numbers will be put after duplicate backups. With no "/" at the beginning, the name will be relative to $PWD.
Var: cgDirEtc="etc"
Templates and other doc related files are put here. Initially they are copied from opt/libre-bib/etc. The files are copied to cgDirBackup if a command would change any of the files.
Var: cgDirLibreofficeConf="$HOME/.config/libreoffice/4/user/database/biblio"
This is the location of Libreoffice's bibliography DB connection information. The path might need to be changed for different Libreoffice varsions.
Var: cgDirStatus="status"
When a command updates a file, a date-stamped status file is created in the cgDirStatus directory. If a dependent file has a newer time than it's corresponding status file, then the update command will be run.
Deleting all the files in the cgDirStatus dir will force all of the commands to run. That is, they will not check to see if things are newer.
Var: cgDirTmp="tmp"
Temporary working files are put in this dir. This is usually relative to $PWD. If set to an absolute location, be sure there is space and that it is unique across all users and bib processes that could be run. For example, do not define it to "/tmp" because when you run "bib clean" that would remove all files and dirs in /tmp !
Var: cgBackupNum=10
This variable defined the number of backup files or tables to be kept. This can be set to 2 to 100.
Var: cgDbHost="127.0.0.1"
Usually this will always be set to the localhost IP. That works better than using a name or localhost.
Var: cgDbName="biblioexample"
This is the name of the database.
Var: cgDbUser="$USER"
This is the name of your DB user. Typically it is the same as your login user name, but you can used any name.
Var: cgDbPassHint="b4n"
This will be shown when you are prompted for the DB User's password.
Var: cgDirCache="$cgDirTmp"
Sensitive files, such as passwords, will be put in this directory. For good security this directory could be on a mounted USB thumb drive, or an encrypted disk. If this is not found, programs that need DB access will not continue.
Var: cgDbPassCache="$cgDirCache/.pass.tmp"
When you use commands that need to connect to the DB you will be prompted for the user's DB password. It will be saved here. It is not encrypted, so don't use the DB User/Pass for sensitive DBs.
Var: cgDbPortLocal="3306"
This is the port for the DB, on the system where the DB is running.
Var: cgDocFile="example.odt"
This it the whole reason for this app and hopefully this shows why you went through the work of creating the biblio.txt file.
This is your Libreoffice document file that contains bibliographic references. {REFs}
Var: cgLoFile="biblio.txt"
This is the text file you will use for adding and updating bibliographic entries. This is much easier to manage and backup than using the DB for everything.
biblio.txt and biblio-note.txt are the files you will be editing the most. biblio.txt is where you will be putting most of the bibliographic information about a book, article, web page, video, etc.
The key.txt file just gives some quick tip on the kind of values you can put after the Tags. It isn't used anywhere else, so you can edit or delete the file.
Var: cgDbTblLo="lo"
This is the name of the primary LibreOffice bibliographic DB table.
Var: cgDbTblBib="bib"
When the lo table is updated this table is created to do some simple formatting, so the bibliography will not be cluttered with duplicate commas when there are empty values.
Var: cgBackupFile="$cgDirBackup/backup-lo.csv"
If you run the backup-lo command this is where the backup will be put. If there is already one there, then that will be backed up.
Commands are entered after "bib". For example: "bib setup-bib"
Just type "bib" for a quick list of commands.
To bring up this manual in your default browser, type: "bib help"
More than one command can be used. They will be run in order. Execution will stop at the first command that fails. ("ignored" errors are OK.) For example, you have modified some entries in biblio.txt and you have modified your libreoffice document with these new references. You could run: "bib import-lo ref-new ref-update"
If a command doesn't run, it could be the files are up-to-date. If you want to force a command to run, remove the status/ file associated with the command, or "touch" one of the dependent files. For example: import-lo will only run if the biblio.txt file is newer than the last time import-lo was run. To force it to run: "rm status/import.lo.date"
After editing the conf.env file, run this command to define the directories and initial files, in the current directory.
Run this to setup and verify the DB connecton is working.
Run this to do a quick check that everything is correctly defined.
This will run: $EDITOR $cgDocFile &
Import any changes to $cgLoFile (biblio.txt) to the lo table. The lo table will be backed-up in the DB.
Export the lo table to $cgLoFile (biblio.txt). The existing biblo.txt file will be backed up to $cgDirBackup.
Backup the lo table to a cvs file in $cgDirBackup.
New biblio {REF} tags have been added to your odt file. Run this command to update your odt file with the current biblio entries found in the lo table. If there are no new entries, the file will be unchanged.
If the file is changed, the original file will be found in the backup/ dir. So your odt file can be restored if there are problems.
If the lo table has been updated with different values, then run the ref-update command.
Internal: see /opt/libre-bib/etc/cite-new.xml for the template that will be used.
This will format the entries with the "Endnote Characters" style, and insert the non-empty bib-field values.
If the lo table has been updated with different values, then run this command to update the odt file with the new values. This command will not modify any new {REF} tags.
The original file will be found in the backup/ dir. So your odt file can be restored if there are problems. It could be there are no changes to the file, but this command doesn't check for difference, it just replaces all of the biblio-entries it finds in the odt file.
Internal: see /opt/libre-bib/etc/cite-update.xml for the template that will be used.
This will only update non-empty bib-field values. The style won't be touched.
After setting the bibliograpy entry styles, run this command so the style is saved. That way it can be restored, if the odt file is recreated.
Replace the bibliograpy entry styles from the styles saved with save-style.
Report on the size of tables and files. Also list what needs to be updated.
Remove temporary files. This will also remove the DB password cache file.
Report the current bib version. Include this in any bug reports.
Run: $EDITOR biblio.txt &
Show the libre-bib manual in a browser window.
DB Tables: If a table exists and cgBackup is "true", then the table will be copied to the table name with a datestamp (YYYY-MM-DDHH-MM-SS) appended. For example, bib -> bib2023-04-0214-18-37
Files: If a file exist and cgBackup is "true", then the file will be copied to FILE.bak. If the .bak file exist then a ".~N~" will be appended after that (larger Ns are more recent).
Backup cleanup: run TBD????, it will prompt to confirm deletes of backup tables or files.
To restore a table. In mysql, follow this example:
drop table `bib`; RENAME TABLE `bib2023-04-0214-18-37` TO bib;
If you are managing multiple bibliographies, you might have some common settings.
If you change the /opt/libre-bib/etc/conf.env file, you will need to rebuild some things. Here are the steps:
cd /opt/libre-bib/etc edit conf.env bash -n conf.env # syntax check cd BIB-PROJECT # any of your bib project dirs bib rebuild # update user default file, and conf.php
Source /opt/libre-bib/etc/conf.env and conf.env in a bash script call your own Makefile, other bash scripts, or php scripts to run things. Your php scripts could include /opt/libre-bib/etc/conf.php to define the ENV vars as globals, or just use $ENV['cgVarName'].
doc/example/example-outline.org
This assumes you have everything installed and working. This will use the example files.
$ cd $HOME $ mkdir -p project/biblio $ cd project/biblio $ bib
Usage:
bib [-n] Cmd
Cmds:
import-lo, export-lo, backup-lo, restore-lo
ref-new, ref-update
status, setup-bib, clean, connect, version, help
$ bib help
Error: Missing conf.env, copying it now
Edit conf.env with your details. Uncomment the ones you are changing.
Then run: bib setup-bib
Usage:
bib [-n] Cmd
Cmds:
import-lo, export-lo, backup-lo, restore-lo
ref-new, ref-update
status, setup-bib, clean, connect, version, help
$ ls
conf.env*
If you accidentally ran bib in a directory that is no going to be a bibliography directory, just delete the conf.env file.
$ emacs conf.env
change:
export cgDbPassHint="b4n"
export cgDbUser="$USER"
to
export cgDbPassHint="fav-pet"
export cgDbUser="example"
save, and exit
$ bib setup-bib
Missing example.odt. Copy an example from /opt/libre-bib/doc/example/example.odt Missing: biblio.txt. Copy an example from /opt/libre-bib/doc/example/biblio.txt
$ ls
backup/ biblio.txt conf.env~* key.txt status/ biblio-note.txt conf.env* example.odt tmp/
$ bib import-lo
Problem with setup: SQLSTATE[HY000] [2002] Connection refused make: *** [/opt/libre-bib/bin/bib-cmd.mak:100: status/import-lo.date] Error 1
Open another terminal:
$ ssh myserver
Enter passphrase for key '/home/bob/.ssh/id.mysys': bob@mxlinux:/home/bob $
Minimize the terminal window.
$ bib import-lo
Nothing was output. Edit conf.env and change cgVerbose to true.
$ bib import-lo
/opt/libre-bib/bin/import-txt-2-lo.php -c Verbose is on. Backup is on. Problem with setup: Missing: cgDbPassCache tmp/.pass.tmp. To set it, run: bib connect [89] make: *** [/opt/libre-bib/bin/bib-cmd.mak:100: status/import-lo.date] Error 1
$ bib connect
read -srp 'Password (fav-pet)? '; \
echo $REPLY >tmp/.pass.tmp
Password (fav-pet)? First define tunnel: ssh HOST.example.com
See: /home/bob/ssh/config
show databases; use DBNAME; show tables;
if [[ "true" == "true" ]]; then \
tPort=3308; \
else \
tPort=3306; \
fi; \
mysql -P $tPort -u example --password=$(cat tmp/.pass.tmp)
-h 127.0.0.1 biblio_example
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 784
Server version: 10.5.18-MariaDB-0+deb11u1 Debian 11
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [biblio_example]> quit
Bye
(Clearly I need to cleanup the outputs.)
$ bib import-lo
/opt/libre-bib/bin/import-txt-2-lo.php -c Verbose is on. Backup is on. . Processed 292 lines. [263] Inserted 31 records. [264] /opt/libre-bib/bin/convert-lo-2-bib.php -c Verbose is on. Backup is on. Processed: 31 [221] date +%F_%T >status/import-lo.date
This imported the biblio.txt file, creating the "lo" table. You can run "bib connect" and use sql commands to look the table. For example:
show tables; show fields from table lo; select Identifier,Booktitle from table lo;
$ bib backup-lo
cp: cannot stat 'backup/backup-lo.csv': No such file or directory ............................... Processed: 31
$ ls backup/
backup-lo.csv
$ bib ref-new
Unpack example.odt [319] Start processing [292] Processed 1056 lines. [303] Found 2 references. [304] Backup example.odt [339] Final clean-up with tidy [343] Repack example.odt [354] Done. [386]
This updated the REF tags so they are now biblio entries. You'll also see the original example.odt was copied to the backup/ dir.
Run: libreoffice to see how they have changed.
$ libreoffice example.odt
If you run import-lo with updated entries, then run ref-update to update them in the example.odt file. If you add new REFs to the document then you would run ref-new again.
$ bib ref-update
Verbose is on. Backup is on. Unpack example.odt [330] Start processing [303] Processed 1065 lines. [314] Found 2 references. [315] Backup example.odt [350] Final clean-up with tidy [354] Repack example.odt [365] Done. [396]
Now you can add the Bibliography to the end of your document, and setup the styles for the different Type of entries.
TBD
See the Makefile in github project.
The best source for the maps can be found in bin/util.php.
This has some minor differences when looking at the field in the Bibliography style section.
This maps the lo text file Tag names to the lo-table field names.
Do some simple formatting of the lo-table values and put them in the bib-table, so that the Bibliography style is easily setup.