Migrate Joomla! 1.5.x to 2.5.x+

: Category: Joomla; 30 January 2011; Hits: 192999

Update June 2013 - How to use this migration script:

Make a backup copy of your old Joomla website (1.5.x)
Download and install the latest Joomla CMS
- Go through the Joomla Web-based Installation GUI until the process deletes the Installation folder.
- Avoid installing Sample Data if possible (there will be some but this script has been reworked to potentially accommodate).
Download my migration script
- Decide on which script to use:
  1. If you have a small site (less than 10'000 articles), then you will use the single script "migrate_j15_to_j25_new_v1_8.sql"
  2. For larger sites (~200'000 articles), then you will need to use the 3-part script "migrate_j15_to_j25_new_v1_6_pt1.sql", "migrate_j15_to_j25_new_v1_6_pt2.sql", "migrate_j15_to_j25_new_v1_6_pt3.sql". Part 2 is dedicated to content migration, the commands are split to do about 5000 articles on each iteration so as to avoid session timeouts and memory issues.
- Edit the script, renaming the database and table names used (DO NOT CONFUSE THE FOLLOWING DATABASE NAMES OR YOU WILL END UP WITH NEITHER SITE WORKING!!!)
  1. Open the SQL file in your favorite text-editor
  2. Rename any instance of "my_old_database.jos_" to the name of your old Joomla 1.5 database or a copy of (note the "jos_" prefix to all J15 tables, if you have changed this then reflect this in the SQL file).
  3. Rename any instance of "my_new_database" to the name of your new Joomla 3.0 database (don't change the table prefix for the new Joomla and instead do this as the next step).
  4. Rename any instance of "my_new_database.my_prefix_" to the name of your new Joomla 3.0 database (note that we're reflecting the prefix to use but also the database name - by this step, "my_new_database" should have been renamed).
  5. Save the SQL file, this is now unique to your website and this process. It does not contain usernames or passwords but has the real database names and all website content, so do not redistribute without taking security measures.
Run the SQL against your new database
- Open your favorite database management tool (phpMyAdmin, workBench, navicat, sqlyog?) and connect to the new database.
- This script will use both databases, reading from the old Joomla database and writing to the new one.
- To run this script, your user needs SELECT, INSERT and UPDATE privileges on the new database (only SELECT on the old database will do). Special care has been made to avoid the use of functions, stored procedures, temporary tables, etc. This allows broader compatibility and customers who do not have DBAs to run this themselves. Personally I grant the user who will be executing the script full privileges or at least the same as needed to install Joomla CMS.
- Cross your fingers and use your DB management tool to run the SQL file (or copy the contents of the SQL file into the input field)
Re-build/Auto-correct some Joomla inconsistencies
- Login to your Joomla Admin Panel using a System Administrator account ("Super User"?)
- Go to Categories > click on the "REBUILD" icon (top right)
- Go to Menus > click on the "REBUILD" icon (top right)
Check your Joomla Admin User
- Each new Joomla version is trying a different method making this very diffcult to maintain. Check that the "super administrators" for Joomla 1.5 are in your new Joomla site (as "Super Users").
- Check that the system administrator for the new Joomla site (created on installation) is still a "Super User" (e-mail address, receives sys emails) and note their ID (identifier number).
- See if elements created on install show as created by the admin user and not by one of your users who coincidentally has the same identifier.
- Check that your OddUser (will be last user listed in users table) can login and their respective articles are still attributed to them.
Prepare for Go Live
- This migration script is not a silver bullet. It was created to take away the hours I know I would spend without it.
- This is not an upgrade; it is likely that your client may reconsider the template they are using or may want to add new 3rd-party components.
- If asked: "how long will it take", do not say 5 minutes. This script runs on average in 15 seconds. The remaining hours, days, weeks will be spent on bringing your website up to standard with all issues addressed and sporting the latest version of the Joomla CMS!

UPDATE 2012 - Valid for Joomla 2.5.x but only experimental for Joomla 3.0.x!
I still do not recommend using this for your site if you have no test environment, however... I have used this script continuously to migrate J1.5.x websites to J2.5.x without issue. I have managed to do whole migrations in under 2 hours (this script runs in about 20 seconds based on 1000 articles and 5000 users) and my clients are satisfied! I am so pleased I made this script; it has saved me so much time!

What does this do?
After many tweaks and corrections having upgraded multiple sites in the past few years with this script, the version below migrates the following:

Articles
Banners
Categories & Sections
Menus ^{* NEW}
Messages
Newsfeeds
Users ^{* UPDATED}
Weblinks

So this is fun. We've waited and waited and Joomla 1.6 is now stable... Exciting because this is not a simple 1.5.21 to 1.5.22 upgrade. But don't believe the drama across the web, the system introduces only a few new fundamental improvements (particularly Access Control and Groups) which actually only means some minor database alterations.

I've googled, yahood and binged, but still can't find a good way to upgrade my site from Joomla version 1.5 to 1.6 so as usual I find myself writing the first migration script. Why use a script? Well apps like jUpgrade didn't work for me. This is how to manually migrate using a MySQL database management tool like PhpMyAdmin. I'm not too bothered about upgrading my own personal site (Joomla 1.5), it's just the silly number of client websites I have to upgrade.

This I find a lot simpler, my day job involves working with SQL so this script was rather basic and quick to write. You have two databases, one with your Joomla 1.5 old content, and one following a fresh install of the latest Joomla (at time of last print: 2.5.6 - preferably no sample data pre-installed). Change the database names in this script (including the new random prefix for J2.5) and run it. 20 seconds later your Joomla 1.6-2.5.x website will have all your J1.5 articles and users...

DISCLAIMER:

This is to help ME migrate Joomla 1.5 sites to Joomla 1.6 automatically. I have also been using this happily for Joomla 2.5.x websites and more recently for an experimental Joomla 3.0.x website.

I do not work for Joomla!

This script is for my own use.

Joomla! are in no way liable for this script.

You can copy this script as long as you specify a link to its source.

You can use this script as long as you don't hold me responsible for the results.

If you don't understand what this script is doing, you shouldn't be running it. Please ask someone who does.

copy raw

joomla Fatal error: Call to a member function getPath categories.php on line 435

joomla Fatal error: Call to a member function getPath categories.php on line 435

« DOWNLOAD »

Further amendments via the GUI

CATEGORIES
1. Go into Content > Category Manager > Rebuild
2. Organize the category structure.
USERS
1. Re-assign "Super Administrators" to "Super Users"
MENUS
1. Should work but if not click on "Rebuild"

Issues I ran into

joomla Fatal error: Call to a member function getPath categories.php on line 435

I haven't found a way to automate this yet but you can fix all of these in one go by logging into your Joomla! 1.6.0 admin panel > content > category manager > rebuild (icon in top right).

« DOWNLOAD »

ChangeLog (for this SQL script):

2013-06-04: Fix for odd user (user in Joomla 1.5 which accidentally has ID of new admin in Joomla 3.0) - associates article content if ID and Username match.
2013-03-13: Performance tweaks. Moved to downloads section of website.
2012-11-19: Corrected user migration (new admin is determined by first user in new users table).
2012-11-15: Additional corrections for Menu migration.
2012-11-12: Corrected menu migration if menu items already exist in new site.
2012-11-08: Corrected some menu migration. J15 admins now admins in J30.
2012-11-02: Added menu migration to the script.
2012-11-02: Upgraded to account for pre-installed content.
2012-11-02: Upgraded to work with both Joomla v2.5.x or 3.0.x.
2012-07-25: Fixed bug in older versions of MySQL reserved word "name".
2012-07-24: Fixed bug of new admin not being allowed into admin panel.
2012-03-25: Changed instructions regarding changing table prefixes.
2012-03-21: Added "Update 2012" to explain compatibility with Joomla version 2.5.6.
2011-01-30: First draft of this script

Still To Do

Add intro script to automatically check I don't get the database names mixed up.
Get the menu and category rebuild scripts to run automatically.
Menu item issue: Confirm if J15 Sections not linking automatically (eg. "Joomla CMS")
Menu item issue: Weblinks component not linked to same category as in J15.
Menu item issue: re-examine why "Contact Us" link moves one less order than in J15.
Look to migrate categories, blog layouts, article settings to J30 equivalents.
Menu item issue: fix migrate user registration, user profile menu items.

« DOWNLOAD »

Category: Joomla :: Article: 319

Comments

Not rated

Nicola

9 years ago

Hi,
back to the 90's

For those who encounters en error like:

[Err] 1062 - Duplicate entry for key 'idx_client_id_parent_id_alias_language' in prefix_menu table
(step II.12.2 line 1083

the error is caused by J15 with duplicate menu names/alias.
I had duplicates aliases placed in different menu groups, that is not possible in J2.5/3
- home
- home
- contact
- contact

So I renamed them to
- home1
- home2
- contact1
- contact2
and so on.

After that, Import was successfull.
Probably SEF is somewhat compromised by changing aliases, but menu structure is imported and navigation preserved.

Hope this helps.
Great script Joel! Thanks.

Reply | Reply with quote | Quote

webtrebol

Thank you a lot Joel! It did its job migrating a Joomla 1.5 content into Joomla 3.5.1. I had some troubles because of some duplicated items. I would like to give an advice to all the persons that will try this script: CLEAN your old installation from ALL unpublished items and the script will work perfectly

lliseil

10 years ago

* Categories:
- duplicates, I suppose from the multiple times I launched the script.
- love the notes let by JMS in the 'Alias' field: 'Note : Import from J15. PreviousID=catid141. SectionID=46) '
- client used flexicontent in J 1.5 to includes subcategories; this structure is of course not preserved (all subcategories to the same level)

* Articles:
- all there; no duplicate.

Both assetweaver and assetdiagnostic show multiple Asset errors for Categoris and Articles.

* Contacts:
- all there
- duplicates

* Links
- not migrated

* Banners and Banner Categories
- are there
- duplicates that I can trash, but can't empty unless opened and saved.

Thank you very very much for sharing JMS.

View replies

WebmasterLegacy

Interesting report LLiSEIL! I'll definitely have to review it when Joomla 3.5 comes out.

Error 1: is usually bypassed by simply removing the |joe| marker from all rows of that column. I do this when the client already has content.

Error 2: The menus as I recall were the most challenging part to migrate. I like that Joomla 1.6+ has a rebuild button in the admin interface which I do recommend.

Error 3: I could not recreate the error but it sounds as if it added all the articles and then conflicted with itself. Are the article IDs the same in both sites (they don't need to be but that would answer what the database system did with them).

Categories: another challenging part as 1.5 had sections. Again I recommend the rebuild button Joomla has for categories.

I can imagine Asset errors as my script had to copy the system Joomla was using. I didn't try copying the last phase that I thought the rebuild did. It's something I will consider in the next review of the script.

Did the weblinks not migrate or were there no weblinks in the old site? As I recall, there weren't many changes from Joomla 1.5 to 1.6 re weblinks. They ought to work the same way as articles.

Duplicates will definitely appear if the script is run several times. Unfortunately, I'm used to creating several copies of the clients Joomla database and merely modifying the script until I'm confident there are no more errors and that it runs in one go.

Thanks for your time and again for this report!

SQL ERror 3
> -- Step 2) Import articles from J15 and insert as articles in J30 INSERT INTO `newbase_j25`.dv46z_content ( id, asset_id, title, alias, introtext, `fulltext`, state, catid, created, created_by, created_by_alias, modified, modified_by, publish_up, publish_down, images, urls, attribs, version, ordering, metakey, metadesc, access, hits, metadata, `language`) SELECT id, 0, title, alias, introtext, `fulltext`, state, catid, created, created_by, created_by_alias, modified, modified_by, publish_up, publish_down, images, urls, IF( attribs='', '', CONCAT( '{"', REPLACE(REPLACE(attribs, CHAR(10), '","'), '=', '":"'), '"}' ) ), version, ordering, metakey, metadesc, CASE [...]
> MySQL a répondu:
> #1062 - Duplicate entry '147' for key 'PRIMARY'

x3

Migrated with JMS:

1. Contents and Structure

* Extensions » Database check is fine.

* Section:
yeap.
[...]

I removed the banner part from the script (has no use for that site) and relaunched JMS:

SQL Error 2
> INSERT INTO `newbase_j25`.dv46z_menu ( menutype, title, alias, note, path, link, type, published, parent_id, `level`, component_id, browserNav, access, params, lft, rgt, home, `language` ) SELECT ( SELECT MAX(b.menutype) FROM `ausoleil_j25`.dv46z_menu_types b WHERE b.menutype LIKE CONCAT(a.menutype, '%') ), a.`name`, CASE WHEN a.`alias` IN ( SELECT DISTINCT b.`alias` FROM `ausoleil_j25`.dv46z_menu b ) THEN CONCAT(a.`alias`, ( SELECT DISTINCT CASE WHEN COUNT(b.`alias`)>0 THEN a.`id` ELSE '' END AS myCount FROM `ausoleil_j25`.dv46z_menu b WHERE b.`alias` LIKE CONCAT(a.`[...]
> MySQL a répondu:
> #1062 - Duplicate entry '0-1-activites-*' for key 'idx_client_id_parent_id_alias_language'

I just retried:
[...]

Hi, I runned JMS on a Joomla 1.5.26 associative website with a few hundred categories and articles, that uses Flexicontent for subcategories. Migrated to blank default Joomla 2.5.28.

I'd like to hear your advice about the way I handled the three SQL errors met (below) while running JMS, as well as about the database future consistency considering these three errors. I'll check it with assetweaver.php in a moment.

SQL error 1
> -- Step 5) Remove script markers from assets.
> UPDATE `newbase_j25`.dv46z_assets b,
> `ausoleil_j25`.dv46z_categories a SET b.`title` = TRIM( SUBSTR( b.`title` , 1, LOCATE( ' :|joes|', b.`title` ) ) ) ,
> b.`name` = CONCAT( 'com_banners.category.', a.`id` ) WHERE b.`title` LIKE '% :|joes|%' AND b.`id` = a.`asset_id` ;
> MySQL said:
> #1062 - Duplicate entry 'com_banners.category.10' for key 'idx_asset_name'

Migration stallled at this point.
[...]

Jessi

11 years ago

Thank you very much for this informtions.I like it...Joomla Video player

dorakura

12 years ago

Oh, silly me! I thought the curly bracket syntax was part of SQL. Well, if metadata is assigned this string as its value, it's non-NULL, right? Why do I get the error message then?

Hi dorakura,

The default value is set at the database level. So if you can access the structure of the Joomla2.5 #__content table, you can check what the default value is. Saying this, this is the default installation.

I can only suggest switching off the strict mode for this database. What is the storage engine you are using? For Joomla Sites my InnoDB is set to STRICT MODE = OFF.

I think you can check by typing the following to the database:
Code:


SELECT @@GLOBAL.sql_mode;
SELECT @@SESSION.sql_mode;
SELECT @@sql_mode;

MyISAM should be fine if thats causing it...

Otherwise it could be some syntax in the code but I've just run this again and it does what it was told to do.

Thank you for your prompt reply. I think you misunderstood Step 2 for Part 2, perhaps. It's the second block of code. The relevant line is this:
Code:


 '{"page_title":"","author":"","robots":""}' AS metadata,

I'm not familiar with this curly bracket syntax. How do I add a default value? Something like default:"" ?

Hi dorakura,

The curly bracket syntax is the way that Joomla 2.5 is storing parameters and if you look up PHP's function serialize (format for storing an array as a string), this may explain things.

The way I interpret it is that the first value is the parameter name and the second value is the data. Each parameter separated by commas and the overall fieldset surrounded by curly braces.

ie.
Code:


{"page_title":"","author":"","robots":""}

page_title=""
author=""
robots=""

Hope that helps!

Thank you very much for your script. It worked great for one site, but am having trouble with another. In Step 2, I get this error:

#1364 - Field 'metadesc' doesn't have a default value

I'm migrating from 1.5 to 2.5. Any ideas? Thanks!

Hi dorakura,

Thanks for your message. I get this error when I try to put a NULL value into a column that does not allow NULLs. Check your #__content table to see if a) the column "metadesc" allows NULLs, b) if the server is running in STRICT mode.

Note that I would NOT disable strict mode if it is running, not without testing every other website running off that server.

One fix is to specify a default value for that column (eg. " ") so set this to a space or 0.

Another fix is to change the SQL script a tad for the Metadesc part (Line 956):
Code:


INSERT INTO
...
SELECT
 id, 0, title, alias, introtext, `fulltext`, state,
 catid, created, created_by, created_by_alias, modified, modified_by,
 publish_up, publish_down, images, urls,
 IF(
 attribs='',
 '',
 CONCAT(
 '{"',
 REPLACE(REPLACE(attribs, CHAR(10), '","'), '=', '":"'),
 '"}'
 )
 ),
 version, ordering, metakey, metadesc,
...

to
Code:


INSERT INTO
...
SELECT
 id, 0, title, alias, introtext, `fulltext`, state,
 catid, created, created_by, created_by_alias, modified, modified_by,
 publish_up, publish_down, images, urls,
 IF(
 attribs='',
 '',
 CONCAT(
 '{"',
 REPLACE(REPLACE(attribs, CHAR(10), '","'), '=', '":"'),
 '"}'
 )
 ),
 version, ordering, metakey, IFNULL(metadesc, ''),
...

Hope that helps!

Joe

Keith Otis

Joel,

Thanks for the great script. I had to change a couple of things to get it to work.

migrate_j15_to_j25_new_v1_7.sql had the old sql syntax of <> in the step to add users. changed those to , and that now works.

Our old J1.5 uses some additional usergroups that were added manually to the database, "Owners" and "Board".

It seems the script is was not updating the usertype in the newdatase.prefix_users. I tried adding usertype to the select and insert of users, and that leaves me with only some of the users correct. Not even all the "Registered" show up in the new db.

I also tried adding more sections to the add groups, that's not working either.

Any ideas?

Hi there Keith,

Thanks for the comment, that was one issue I didn't really get to test, so thanks for bringing it up. It's made me double-check the files available for download and I made the mistake of using the code required to display the SQL on this HTML page. So yes the < > signs need fixing.

As for the issue on different named groups, you can have custom ones, but then you need to add them to the new Joomla 2.5.x+ site before running this script... then...

Find the part:
Code:


 WHEN a.group_id=(
 SELECT id
 FROM `my_old_database`.jos_core_acl_aro_groups
 WHERE `name`='Super Administrator'
 LIMIT 0,1)
 THEN (
 SELECT id
 FROM `my_new_database`.my_prefix_usergroups
 WHERE title='Super Users'
 LIMIT 0,1)

Copy that and put it underneath it. Just change "Super Administrator" into the name of J15 group and "Super Users" into the name of J25 group.

(I used if...then...else rather than script autodiscovering the names because of all the languages in the world and this was easier for my client)

Hope that answers your questions!

I'll release an update shortly with the fix you mentioned.

Joe

drew

Joel,

Thanks for the newly updated instructions. I was nearly successful in getting everything working. We had a custom built 1.5 Joomla template making it difficult to convert. I can attest that all the data looks to have come over. I would add to your article that if you are copying over other data folders from the old Joomla install that you will want to go to Extension Manager>Discover and install the old extensions once the data is copied into place. I found that copying the libraries folder into the new Joomla install will break it, so avoid copying in that folder (from my testing.) I'm going to go back and attempt this on a 2.5 Joomla now and see if I can get that template working. 3.1 is too new for it and I've attempted to make a bunch of corrections to the template code to fix it. I did however get the template to show up in the templates section, but no styles show up, and when attempting to install the template it starts to process and gets a server 500.

Hi Drew,

I've got a silly question, are you sure the template you are using has been upgraded or designed for Joomla 2.5? As they are functionally different then in Joomla 1.5.

If it's a 3rd-party one then hope the suppliers have an updated version otherwise there are quite a few pages via google on upgrading a Joomla template from 1.5 to 1.6. I noticed even further changes to make if you want to make it compatible with Joomla 3.1.

I might add the extension manager > discover point you made but I think I wrote the above during Joomla version 1.6 and their discover button wasn't stable enough at the time to be considered reliable (still don't for non components).

Making a little progress here. I already had a copy of the live site running on this test server. So, I setup a second virtual host in apache, and pointed to my joomla3.1 install path which is running on the same server as the live site copy. I created the 2nd db so it was ready for joomla 3.1 to insatll. I edited the script so that it matches up with the 2 databases properly. I then ran through the joomla3.1 install and removed the install folder. I see articles from the old database. I notice that places where there is old articles that have content such as images or attachments are missing. How should I migrate in the rest of the old content to complete the conversion?

Thanks for the help Joel, your last comment got me moving forward faster than I had for about 2-3 hours the other day :)

-Drew

Hi Drew,

Good point. You need to copy your images folder from your old site over to your new one. You should find the missing images is because it is searching relative to the website you are on. If you have 3rd-party components then you should install these on the new site following the usual process.

I added some much needed instructions to this article following your comment yesterday as I get the question a lot "how do i use it?".

The script above only does the database (data and configuration). I advise installing the new Joomla because that will have the correct file/folder structure rather than trying to correct a Joomla 1.5 to fit in a later version. Your most important file being the "configuration.php" file in the root of your Joomla site and should never be copied to other sites (for example, I can copy the database and all files to a test system which holds a different configuration.php file and this will be an exact website copy).

Please let me know where to begin with this, I've got a joomla 1.5 install and trying to migrate to 3.1.

Do I need to have a new mysql database with data in it, because I have a 1.5 joomla mysql db as my old db and a new mysql db? When I try "mysql shell> source .sql" it throws errors saying the tables don't exist in the new table. Also do I throw the new joomla files over the top of the old ones to complete the upgrade. I didn't see any mention of how to run the sql in the documentation. I was assuming I need to source it since it's a sql script. Let me know what I'm doing wrong.

Hi Drew,

Yes you need to setup two versions of Joomla to use this script. Well one if you don't count your old Joomla 1.5 site. Personally, I'd have a copy of the old Joomla 1.5 site up and running. Search my site for "setup a copy of your Joomla site" if you're unsure on this.

The biggy in using this script is that you need to download and setup the latest Joomla (go through the Installation GUI until it deletes the Installation folder). For less risk and errors, don't include any sample data in the new site. It should be blank and running alongside your old site.

The migration script can then be used, you will need to rename the databases and respective table prefixes to use. Once this is done, run the script against the database and it will migrate the content, users and core Joomla objects.

Try this and if you get stuck, feel free to come back to this post.

I'm a bit confused here. Do I need to put the new joomla install files over the top of the old files? Do I need to have a new mysql database with data in it from the old install? When I tried sourcing the .sql migration script it threw errors because the tables don't exist in the new db. How do you run this .sql file? Thanks for any help you can provide!

-drew

Amílcar Tavares

Hi Joel.

Thank you for this wonderful script! I managed to migrate a website with almost 80000 articles!

I had no problems with the process (MySQL Workbench showed some warnings but no errors) but the website is slower. Could you give me some hints about this?

Thanks.

Hi Amilcar,

80,000 articles is guaranteed to slow down a site. I have had this with other migrations and we concluded that in addition to the number of articles added, Joomla 1.6+ decides that every article has to have an associated asset ("table: #__assets").

In some cases, adding table indexes may help (specifically on the assets table) but adding them to your content will slow any inserts/updates down (ie. "Times Read").

Code:


SHOW INDEXES FROM my_joomla_content;

If you have mySQL workbench (v5.x), enable the Joomla DEBUG mode and try using the database queries for some slow pages and use the EXPLAIN button (next to execute statements/run query).

Other than that, try the standard GZIP page compression techniques provided by Joomla and other third-party developers.

Thanks for your feedback!

roderick

Sorry about my last mail, I fixed it by flushing the trash and just imported categories and articles. Still a lot of work to do but your script helped for sure.
Thnx
Roderick

Hi,
Trying your script to migrate articles from 1.5 to 3.0.

The script stopr with this error:
#1062 - Duplicate entry '2' for key 'PRIMARY'

Catatgories are created. No articles are imported. I deleted all the sample articles and categories up front.

Have you got a suggestion?

Thnx
Roderick

Hi,
Just came over this page Joel. Looking at your script and seeing how long you've been using, improving it and sharing it --thank you for that-- I can't wait to test it!
We used JUpgrade to migrate quite a few sites last year and each caused further work detecting and correcting errors in various tables (mostly Categories). Note that VirtueMart migration itself went fine.
Now I'm planning to migrate to Joomla! 2.5 one of the Joomla 1.5 website that we kept with both your script, J2XML and JUpgrade, and then compare the results. That sure will be teachful and might even be fun!
Please excuse the long message -
ps Can't believe it, already have an account on your site

Hi LLiSEIL,

That sounds great! I only wrote this script because jUpgrade didn't work for me.

The script above will not work on a 200K article. We have found ways of splitting the article content (assuming create function/proc privilege is not granted) is currently the only method that has worked. There's an update to the script above for where we improved performance (having to work with big migrations) which I'll upload as soon as.

Would love to know the results and how this script stacks up. TBH there's a lot of work in upgrading a Joomla 1.5 site and the migration with my script is just a small part of it.

Thx for your message.

Perina

Hi, your scrip runned flawlessly, all articles imported - I can see them listed in backed of 3.0.2, however on backend open says : SQL=SELECT id FROM jos_assets WHERE alias = 'root'

same goes for all articles & categories

Hi Perina,

I haven't done too much with Joomla! v3.x but have you tried the menu and category "rebuild" buttons (using Joomla admin panel)? They help with the left/right parent structures.

Zorro

I got following error on migrating categories - first step of migration:
Column 'parent_id' cannot be null

Marc Gagnon

HEllo Joel. I am looking at your script and maybe you could help me. I want ot create 200 000 articles right in the _content table. However I must take into account the _asset table. I can not figure it out. What would be the script to do so: create 200 000 entries in _content while _asset is being update accordingly. I just need the title and alias created, the rest I fill it up using excel .

Hi Marc,

Yes you need to create the assets along with the articles. Don't forget to associate the asset category and asset articles as well:

Taken from my script above:
Code:


-- -----------------------------------------------------------------------------
-- Step 1) Import articles from J15 and insert as assets in J30

SET @new_jos_assets_last_inc=(
 SELECT (SUBSTRING_INDEX(`name`, '.', -1)*1) AS lastassetid
 FROM my_new_database.my_prefix_assets
	WHERE `name` LIKE 'com_content.article.%'
 ORDER BY lastassetid DESC LIMIT 0,1);
SET @new_jos_assets_last_inc=(
	SELECT IFNULL(@new_jos_assets_last_inc, 0));
SET @new_jos_assets_id_last_lft=(
 SELECT MAX(lft) FROM my_new_database.my_prefix_assets);

INSERT INTO
 my_new_database.my_prefix_assets (
 parent_id, lft, rgt, `level`, `name`, title, rules)
SELECT
 0, @new_jos_assets_id_last_lft:=@new_jos_assets_id_last_lft+2,
 @new_jos_assets_id_last_lft+1, 3,
 CONCAT(
		'com_content.article.',
		@new_jos_assets_last_inc:=@new_jos_assets_last_inc+1
	),
 CONCAT(title, ' :|joes|', id, '|:'),
	'{"core.delete":[],"core.edit":[],"core.edit.state":[]}'
FROM
 my_old_database.jos_content;


-- -----------------------------------------------------------------------------
-- Step 2) Import articles from J15 and insert as articles in J30

INSERT INTO
 my_new_database.my_prefix_content (
 id, asset_id, title, alias, introtext, `fulltext`, state,
 catid, created, created_by, created_by_alias, modified, modified_by,
 publish_up, publish_down, images, urls,
 attribs, version, ordering, metakey, metadesc,
 access, hits, metadata, `language`)
SELECT
 id, 0, title, alias, introtext, `fulltext`, state,
 catid, created, created_by, created_by_alias, modified, modified_by,
 publish_up, publish_down, images, urls,
 IF(
		attribs='',
		'',
		CONCAT(
			'{"',
			REPLACE(REPLACE(attribs, CHAR(10), '","'), '=', '":"'),
			'"}'
		)
	),
 version, ordering, metakey, metadesc,
 CASE
		WHEN access=0 THEN 1
		WHEN access=1 THEN 2
		WHEN access=2 THEN 7
	END AS access,
	hits,
 IF(
		metadata='',
		'',
		CONCAT(
			'{"',
			REPLACE(REPLACE(metadata, CHAR(10), '","'), '=', '":"'),
			'"}'
		)
	), '*'
FROM
 my_old_database.jos_content;

Mykah

13 years ago

Hi. It's me again. Justo to be clear, I would like to add something to my previous comment. The error appears to me at:
-- Step 5) Remove script markers from assets.
MySQL said: Documentation
#1062 - Duplicate entry 'com_banners.category.3' for key 'idx_asset_name'

I'm testing the import in two ways: emptying all tables of the new DB, and with the new DB with the sample DB provided by Joomla. This error appears in both.

Thank you for your time!

Hi Mykah,

I have just completed and published an upgrade to the above script. The errors you mention are conflicts due to joomla assets/categories which already exist in your new system. I was under the impression that the updates to my above script accounted for this but every site is different i guess.

The above script updated 2012-11-02 16:30, has been tested on newly installed joomla 2.5.7 and joomla 3.0.1 with default data and brochure data already installed.

I will try to replicate the error you have been getting but i think you need to clear your assets table (which would mean clearing your site and starting again). If I solve it, I will post an update.

Joe

Hi Joel,

It's me again. I used Migrator to go from 1.0 to 1.5 and I'm testing your script with MySQL Workbench 5.2 CE. It lasts for ever when it reaches the following:

UPDATE db.vlhcg_content a, db.jos_content b, db.vlhcg_categories c, db.jos_categories d SET a.catid=c.id WHERE a.id=b.id AND c.alias=d.alias AND b.catid=d.id AND c.extension='com_content'

May I skip this step for now and go back to it later?

Have a nice weekend. Thanks.

Hi Mykah,

The query you have posted indicates that you are on the same database but migrating the data from one table to another. I haven't tested this script if you have only one database available and would strongly urge you to use two separate databases. Saying that, in theory, it should still be possible.

I wrote the above script so you could run sections separately. Though I would recommend migrating articles and users together. Some sections like Menus can be skipped. Categories not though. The SQL you posted associates categories and articles. Doing this manually would involve opening both tables (content and categories) up and manually changing the catid of the J2.5 article to the corresponding category (J2.5 categories).

The only reason I think it's in a neverending loop is because you may have two aliases the same (two categories called the same) or two categories with the same ID (should be prevented by the system but it's possible).

Hope that makes sense.

Hi Joe, thank you for the reply.

Give me hint: it is better to use your script after cleaning entirely the new database or you suggest other approach?

I'll try to resolve the issues you pointed out.

Mykah

Hi Mykah,

I've just migrated a site using the above script (Joomla 1.5.23 to Joomla 2.5.7). The site to migrate to had no sample data though.

I have another to do this afternoon but it will have content already in the upgrade version of the site so it will be a good test for conflicts.

Note that deleting a category using the Joomla admin panel doesn't necessarily clear this from the system as I don't see how Joomla clears it's assets table (from what I see, it doesn't).

I'll try to run a test migration with specific sample data pre-installed to match your error.

Hi Joel!

I'm having the following error:

- ::::::::: REMOVE SCRIPTING NOTES: jos_assets.name
UPDATE joomla257.nzkwi_assets b,
joomla257.nzkwi_categories a SET b.title = TRIM( SUBSTR( b.title, 1, LOCATE( ' :|joes|', b.title ) ) ) ......
MySQL said: Documentation
#1062 - Duplicate entry 'com_content.category.19' for key 'idx_asset_name'

Do you have any sugestions?

Thanks for sharing the wonderful script!

E. S.

I get an error near

-- ::::::::: ASSOCIATE CATEGORY IDS: jos_banners.catid
UPDATE mydb_upgrade.jos_categories a, mydb_upgrade.jos_banners b
SET b.catid=a.id WHERE b.catid=SUBSTR( a.note, LOCATE('. PreviousID=catid', a.note)+18)
AND SUBSTR( a.note, LOCATE('. PreviousID=', a.note)+13)''

Sorry about that, everytime I update this page, Joomla CMS takes out all my "<" ">" symbols so I have to reload this script. I've now got several backups of this script stored in my "cloud" and have to remember to paste that into this article each time I update it. That line should have been
Code:



-- ::::::::: ASSOCIATE CATEGORY IDS: jos_banners.catid
UPDATE mydb_upgrade.jos_categories a,	mydb_upgrade.jos_banners b
SET b.catid=a.id WHERE b.catid=SUBSTR( a.note, LOCATE('. PreviousID=catid', a.note)+18)
AND SUBSTR( a.note, LOCATE('. PreviousID=', a.note)+13)<>'';

chris

Hi, when executing the first SELECT I get the following error message:

#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-graf.m10tx_assets WHERE 'name' LIKE '%.category.%' ORDER BY 'lastcatid' DESC ' at line 2

Hi Chris,

That statement is only assigning a value to a variable. I've noticed in the excerpt you pasted that there seems to be confusion between backticks (`) and apostrophes ('). Note my syntax as I use backticks for reserved words (eg. fulltext is a MySQL reserved word but Joomla CMS decides to use it anyway for something unrelated - full article content -> this has to be escaped as such `fulltext`). Apostrophes are for values so your comment above should be:
Code:


-graf.m10tx_assets WHERE `name` LIKE '%.category.%' ORDER BY `lastcatid` DESC.

or equally
Code:


-graf.m10tx_assets WHERE name LIKE '%.category.%' ORDER BY lastcatid DESC

I had added these as some systems I was migrating to used an older version of MySQL and the script would halt on reserved words that weren't escaped with backticks.

Denis

14 years ago

Hi!

Thank you.

Is it right for Joomla 1.6.1 or i need to migrate to 1.6 first, then upgrade to 1.6.1 ?

Hiya Denis,

I like v1.6.1 but can't really vouch for it as I'm not running a live site through it yet.

For this script I just install a 1.6.1 and migrate the live site. I'm trying to add to it because just running the script only saves a small amount of time in my opinion.