1. NOTICE: If you are using Fabrik and update to Joomla 3.10, you will need to update to Fabrik 3.10. And, if you are using Fabrik, do not upgrade to Joomla 4, we do not have a supported version ready for release. More information on a release date coming soon. Also, please note that Fabrik 3.10 will not install on any Joomla sites less than 3.8.
    Dismiss Notice

Details view shows no user date with row id -1, but does when user id entered explicitedly

Discussion in 'Professional Support' started by fabrik_83, Jul 18, 2018.

  1. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    I don't see it in my inbox, although I get a *lot* of mail, so it may just be buried. What was the subject, or part of the from address I can search for?

    -- hugh
     
  2. fabrik_83

    fabrik_83 New Member

    Level: Community
    Hugh,

    Subject line is :
    File for thread: Details view shows no user date with row id -1, but does when user id...

    from yahoo email.

    Thanks.
     
  3. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    OK, found 'em. Google had helpfully filed that message away in a completely unrelated folder.

    I'll try and free up time to look at this today, but I'm still kind of playing catchup from my vacation.

    -- hugh
     
  4. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    OK, I imported your CSV with no problems - did it the first time to actually create the List (doing an Import from the Lists section on the backend), it correctly created the elements and populated the table. Then I emptied the table, and did another import using the "front end import" (the CSV menu on the list display itself), and it correctly imported the 11 rows.

    I haven't run the script yet, as we need to figure out why you got that "Sorry the following fields in the CSV File are not found in this table" error, because the fields it then lists in that screenshot you gave don't even remotely match what is in the file you sent me.

    Are you sure you are importing the same file you sent me?

    -- hugh
     
  5. fabrik_83

    fabrik_83 New Member

    Level: Community
    Hugh - sorry for the delay but I didn't get any email notification to your post and only saw it now checking the forum. The cvs file is different from the original posted screen shot as I had trashed the original file, but I did try to load the cvs and modified script file I sent you and got the same error message of not "finding the fields in this table". Could be my script file?
     
  6. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    If you turn the csv plugin off, so you are just importing the data, does it work?

    -- hugh
     
  7. fabrik_83

    fabrik_83 New Member

    Level: Community
    Hugh,
    Screenshot of importing without listcvs plugin–different screen but basically the same error message as previously, no? Import is asking to create a new Fabrik element for each value of the csv file. And to be clear, I'm importing from Fabrik:list> View data (list) of user registration form:

    View attachment 17478
     
  8. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    OK, it definitely looks like a line ending thing. You can see where it's not detecting the line ending between username_element and the first 'id' field on the next line (122).

    And, looking at the file you sent me (using 'od' to see the actual hex value) ...

    Code (Text):

    $ od -cv Student_Info10_08132018.csv
    0000000   "   u   s   e   r   i   d   _   e   l   e   m   e   n   t   "
    0000020   ,   "   n   a   m   e   _   e   l   e   m   e   n   t   "   ,
    0000040   "   e   m   a   i   l   _   e   l   e   m   e   n   t   "   ,
    0000060   "   u   s   e   r   n   a   m   e   _   e   l   e   m   e   n
    0000100   t   "  \r   "   1   2   2   "
     
    ... you can see the file is using Windows style \r rather than UNIX style \n for line endings.

    When did you last do a github update? I made a change back in June to add automatic line ending detection to the CSV import ...

    https://github.com/Fabrik/fabrik/co...75458d7#diff-fd720b864bd1addc71ff268ea50dba91

    ... which may well explain why it works here but not on your system.

    So try doing a full github update, and try the import again. I'd suggest deleting the whole list and start again from scratch, creating the list by doing an "Import" in the main List UI on the backend, so it creates the list and table itself from the CSV.

    -- hugh
     
  9. fabrik_83

    fabrik_83 New Member

    Level: Community
    Hugh,
    After replacing CVS Import file with your latest github build and correcting the header names in my csv file to match the field names of my Fabrik fieldnames, import works! – except for one last major issue: It does not create a J!user as it does when using Fabrik form. If I define the element_id in cvs it doesn't show in that column after import, there is no error on import, but when I try to edit one of the new records I get a Joomla message saying it cannot find Juser ID number, the samme id number for that student in the csv file:

    View attachment 17480

    I tried to import the same files without any element_id field and fabrik tried to create a Juser, but actually just used my ID for all the records:

    View attachment 17481

    Feels like we're getting close to getting this–I have about 2,500 users to import once I figure out how to navigate this all.
    If it helps I'll send you the latest version of my csv file and import.php script. Thanks.
     
  10. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    Yup, send them.

    Also, post a screenshot of your listcsv plugin settings.

    -- hugh
     
  11. fabrik_83

    fabrik_83 New Member

    Level: Community
    Here's my listcsv setttings: bare minimum with the rest of the form not shown in screen shot set to default "None selected" and empty code fields.

    View attachment 17482
     
  12. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    Are you sure you sent me the right files?

    The create_client_user_jTest.php you sent me doesn't match the element names in your CSV, it references ...

    Code (Text):

    $csv_user->full_name = 'register_users___full_name';
     
    ... which doesn't exist in the CSV?

    Also userid_elem doesn't exist, but I'm assuming you added that after you created the list initially?

    -- hugh
     
  13. fabrik_83

    fabrik_83 New Member

    Level: Community
    Yes, this is the file that selected in my listcsv plugin settings, which I just checked again. It's in plugins>fabrik_list>>listcsv>scripts:
    View attachment 17483
    The full_name in my fabrik structure is a calc field that concatenates students first and last name field which I did because Juser only has one field for required name field. Works fine when creating Juser with fabrik (my "Registration") form.

    And yes, I deleted the userid_elem column from the csv file to demostrate my last point that when column is not provided it imports all records my Juser id! I'll send you new csv with userid_elem's to simulate no Juser generated.
     
  14. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    Ah, OK, the main problem is that you changed the wrong part of the configuration.

    You did this, changing the actual variable name (the bit to the left of the = sign) as well as the value (the bit to the right) ...

    Code (Text):

    /*
     * REQUIRED
     *
     * The full Fabrik element names for the username, email, name and J! userid.
     * The plugin will write the newly created J! userid to the userid element.
     * These four are REQUIRED and the code will fail if they are missing or wrong.
     */

    $csv_user->userid_elem = 'register_users___userid_elem';
    $csv_user->full_name = 'register_users___full_name';
    $csv_user->prim_email = 'register_users___prim_email';
    $csv_user->user_name = 'register_users___user_name';
     
    Here's what it should be. I've added some text to the comment (which I'll commit to the example in github) to better explain it ...

    Code (Text):

    /*
    * REQUIRED
    *
    * The full Fabrik element names for the username, email, name and J! userid.
    * The plugin will write the newly created J! userid to the userid element.
    * These four are REQUIRED and the code will fail if they are missing or wrong.
    *
    * NOTE - only change the quoted part after the =, like 'changethis___username',
    * do not change the variable name part before the =.
    */

    $csv_user->username_element = 'register_users___user_name';
    $csv_user->email_element = 'register_users___prim_email';
    $csv_user->name_element = 'register_users___student_fname';
    $csv_user->userid_element = 'register_users___userid_elem';
     
    I just ran a test with the correct config, and it works, except for one user it found a duplicate email for:

    The way I did it was to create the List from an import. The edit and save the List settings one time (to work round a "feature" whereby it won't show any controls on the list display till you do that). Then "Empty" the list. Then create the userid_elem element as a simple 'field' element. Then add the listcsv plugin to the List, and do a "CSV Import" from the list view.

    Note that I just used 'register_users___student_fname' (first name) as the name element, as there isn't a full name element in the CSV you sent me. If that's a problem, I could tweak my example code to allow optionally specifying a first and last name element rather than a full name element, and contatenating those together into the full name for the new user. Let me know.

    Last note ... you would almost certainly have to bust your import up into manageable chunks. Creating users is a time consuming business, and trying to do 2000+ in one go would certainly time out. You should experiment with different size imports, and find the number at which your system chokes. Probably start with 50, and go up by 50 at a time till it breaks.

    -- hugh
     
  15. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    Ooops, I just noticed the code is rather old, and isn't setting the default Group ID for the new user correctly.

    I'm adding an option to allow that to be either set in a field in the CSV, or set to a standard default, and to use 2 (Registered) if netiher of those options are used.

    So give me a few minutes to test and commit the changes before you create any users.

    -- hugh
     
  16. fabrik_83

    fabrik_83 New Member

    Level: Community
    Hugh,
    Thanks for the fantastic support. Won't be able to try the new workflow for at least half an hour anyway so don't rush. Regarding added first and last name that would get concatenated to full name in Juser, that would be awesome as you can image the number of same names that would duplicate for 2,500 students!
    Thanks.
     
  17. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    OK, so you'll need to update the example files from github:

    https://github.com/Fabrik/fabrik/commit/b93b975519ecc59f00383fdca48031f97002b328

    Either do a full github update, or just extract those files from the ZIP and upload them.

    And here is the complete code for your plugins/fabrik_list/listcsv/scripts/create_client_user_jTest.php file:

    Code (Text):

    <?php
    /**
    * Fabrik List CSV plugin example script
    *
    * @package     Joomla.Plugin
    * @subpackage  Fabrik.list.listcsv
    * @copyright   Copyright (C) 2005-2016  Media A-Team, Inc. - All rights reserved.
    * @license     GNU/GPL http://www.gnu.org/copyleft/gpl.html
    */

    /**
    * Some example code for creating J! users when importing CSV file.
    *
    * Instructions:
    *
    * 1) Install the listcsv plugin, and add it to the List you are importing to.
    *
    * 2) Copy and rename this file, to whatever name you want, in the same directory,
    * to avoid having your changes overwritten next time you update Fabrik.
    *
    * 3) Modify the REQUIRED 'changethis' options below to match your full element names.
    *
    * 4) Set any of the OPTIONAL options below.
    *
    * 5) On your List plugin settings for the List CSV plugin, select the renamed file from step 2
    *
    * 6) Run your CSV import.  This plugin will run once for each row being imported, and
    * attempt to either create or modify a J! user accordingly. Modification occurs if username already exists
    *
    */
    defined('_JEXEC') or die();

    require_once JPATH_SITE . '/plugins/fabrik_list/listcsv/scripts/csv_import_user_class.php';

    $csv_user = new ImportCSVCreateUser;

    /*
    * REQUIRED
    *
    * The full Fabrik element names for the username, email, name and J! userid.
    * The plugin will write the newly created J! userid to the userid element.
    * These four are REQUIRED and the code will fail if they are missing or wrong.
    *
    * NOTE - if your import has separate fields for first and last name, you can set $csv_user->name element to
    * be blank, like ...
    *
    * $csv_user->name_element = 'changethis___name';
    *
    * ... and instead set first_name_element and last_name_element in the OPTIONAL settings further on.
    *
    * NOTE - only change the quoted part after the =, like 'changethis___username',
    * do not change the variable name part before the =.
    */

    $csv_user->username_element = 'register_users___user_name';
    $csv_user->email_element = 'register_users___prim_email';
    $csv_user->name_element = '';
    $csv_user->userid_element = 'register_users___userid_elem';


    /*
    * OPTIONAL
    *
    * The following are optional:
    *
    * first_name_element and last_name_element - as noted in the REQUIRED settings, you can optionally
    * use these if you import file uses separate first and last name fields, and we will concatenate first and
    * last names with a space to create the full name for the user.
    *
    * password_element - if specified, plugin we will use this as the clear text password
    * for creating a new user.  This value will be cleared and not saved in the table.
    * If not specified, plugin will generate a random password when creating new users.
    *
    * first_password_element - if specified, the clear text password used to create the
    * user will be stored in this field, whether it came from a specified password_element
    * or was randomly generated.  Can be same as password_element if you want.
    *
    * user_created_element - if specified, this element will be set to a configurable value
    * if a user is created.
    *
    * user_created_value - value to use when setting user_created_element above.
    *
    * group_id_element - if specified, plugin will use the value of this element as the (numeric) group ID
    * to assign the new user to.  No real sanity checking is done, so BE CAREFUL not to assign people to
    * things like Super Admins!  if not specified, the plugin will use 2 (Registered, in a default
    * Joomla install).
    */

    $csv_user->first_name_element = 'register_users___student_fname';
    $csv_user->last_name_element = 'register_users___student_lname';
    $csv_user->password_element = '';
    $csv_user->group_id_element = '';
    $csv_user->first_password_element = '';
    $csv_user->user_created_element = '';
    $csv_user->user_created_value = '1';

    $listModel = $this->getModel();

    /**
    * If you want to use a group ID other than 2, but don't want to use a field in the import to specify the
    * group ID to use, you can change the default by inserting the following line (with the desired ID rather
    * than 123) before the createUser() line below.
    *
    * $csv_user->default_group_id = 123;
    */

    $csv_user->createUser($listModel);

     
    If you need to add the new users to anything other than group ID 2 (Registered, if you haven't changed the out-of-box setup), then either add a field to your import and set the optional group_id_element accordingly, or follow the instructions right at the end of that file, to set the default_group_id.

    But as-is, the above file works, using fname and lname to create the full username, and adding them to group 2.

    -- hugh
     
    Last edited: Aug 24, 2018
    fabrik_83 likes this.
  18. fabrik_83

    fabrik_83 New Member

    Level: Community
    Hugh,
    Latest version of listcvs works fantastic except it still inserts my user id into all fabrik user id even though all records are created in Joomla Users with unique Juser ID. I've written a sql query that replaces my id number with the correct, corresponding Juser ID and all is well. Thanks again.

    One more question about importing: Is it possible for me to temporarily change the time-out setting so I can import more records than currently? Is that a Joomla or Fabrik setting? I'll change it back to default after I finished importing.

    Cheers.
     
  19. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Community
    Hmm, strange, works fine here, sets that userid_elem to the newly created J! user ID. If you send me another copy of the import file, I'll try again, see if I can see where it might be going wrong.

    The script timeout is a PHP setting, in php.ini. Which requires editing your php.ini and restarting the web server. Not something you want to be doing.

    You could try adding this line to your create_client_user_JTest.php ...

    Code (Text):

    ini_set('max_execution_time', 300);
     
    ... somewhere right at the start, like after the "defined or die".

    That sets the time in seconds (so that's 5 minutes). That may or may not work, depending on your hosting setup. Typically shared servers won't let you modify ini settings from PHP. But if you have your own VM, it should work ... but it might not, depending how your Apache / nginx is configured.

    -- hugh
     

Share This Page